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PREFACE 


The  work  reported  herein  was  done  for  the  Advanced 
Instructional  Design  Advisor  project  at  the  Air  Force 
Armstrong  Laboratory  (AL/HRT).  The  substance  of  this 
research  was  done  under  contract  to  Mei  Associates,  Inc., 
the  primary  contractor  on  the  Advanced  Instructional  Design 
Advisor  (Contract  No.  F33615-88-C-0003 ) . 

This  work  was  done  as  part  of  the  first  phase  effort  on 
the  Advanced  Instructional  Design  Advisor.  The  initial 
phase  of  this  project  established  the  conceptual  framework 
and  functional  specifications  for  the  Advanced  Instructional 
Design  Advisor,  an  automated  and  intelligent  collection  of 
tools  to  assist  subject  matter  experts  who  have  no  special 
training  in  instructional  technology  in  the  design  and 
development  of  effective  computer-based  instructional 
materials . 

Mei  Associates '  final  report  for  the  initial  phase  will 
be  published  as  an  Armstrong  Laboratory  Technical  Paper.  In 
addition,  Mei  Associates  received  14  papers  from  the  seven 
consultants  working  on  this  phase  of  the  project.  These  14 
papers  have  been  grouped  into  six  sets  and  edited  by  AL/HRT 
personnel.  They  are  published  as  Volumes  1  -  6  of  Designing 
an  Advanced  Instructional  Design  Advisor: 

Volume  1:  Cognitive  Science  Foundations  (AL-TP- 
1991-0007) 

Volume  2 :  Principles  of  Instructional  Design 
(AL-TP-1991-0017) 

Volume  3:  Possibilities  for  Automation 
(AL-TP-1991-0008) 

Volume  4:  Incorporating  Visual  Materials  and  Other 
Research  Issues  (AL-TP-1991-0017-Vol-4 ) 

Volume  5;  Conceptual  Fr2uneworks  (AL-TP-1991-0017-Vol-5 ) 

Volume  6:  Transaction  Shell  Theory  (AL-TP-1991- 
OOI7-V0I-6) 

This  is  Volume  4  in  the  series.  Sections  I  and  IV  were 
written  by  Dr.  Michael  Spector.  Section  II  was  written  by 
Dr.  Alinda  Friedman.  Section  III  was  written  by  Dr.  Martha 
Poison. 
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SUMMARY 


The  Advanced  Instructional  Design  Advisor  is  an  R  £<  D 
project  being  conducted  by  the  Air  Force  Armstrong  Laboratory  in 
response  to  an  Air  Training  Command  (ATC)  Manpower,  Personnel, 
and  Training  Need  calling  for  improved  guidelines  for  authoring 
computer-based  instruction  (CBI)  (MPTN  89-14T) . 

Aggravating  the  expensive  and  time-consuming  process  of  CBI 
development  is  the  lack  of  Air  Force  personnel  who  are  well- 
trained  in  the  areas  of  instructional  technology  and  educational 
psychology.  More  often  than  not,  a  subject  matter  expert  with 
little  knowledge  of  CBI  is  given  the  task  of  designing  and 
developing  a  computer-based  course.  Instructional  strategies 
that  work  in  a  classroom  are  often  inappropriate  in  a  computer- 
based  setting  (e.g.,  leading  questions  may  work  well  in  a 
classroom  but  are  difficult  to  manage  in  a  computer  setting) . 
Likewise,  the  computer  offers  the  capability  to  present 
instruction  in  ways  that  are  not  possible  in  the  classroom  (e.g., 
computer  simulations  models  can  be  used  to  enhance  CBI) . 

The  Advanced  Instructional  Design  Advisor  is  a  project  aimed 
at  providing  subject  matter  experts  who  have  no  background  in 
computer-based  instructional  systems  with  automated  and 
intelligent  assistance  in  the  design  and  development  of  CBI.  The 
goal  is  to  reduce  CBI  development  time  while  insuring  that  the 
instructional  materials  are  effective. 
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I.  INTRODUCTION  (Spector) 


The  Advanced  Instructional  Design  Advisor  (AIDA)  is  an  R  &  D 
project  aimed  at  providing  automated  and  intelligent  assistance 
to  inexperienced  instructional  designers  who  have  the  task  of 
designing  and  developing  computer-based  instruction  (CBI) .  The 
particular  problem  being  addressed  by  this  line  of  research  is 
the  need  for  more  cost  efficient  methodologies  for  the  design  and 
development  of  CBI.  Current  methods  for  developing  CBI  are 
expensive,  time-consuming,  and  often  result  in  ineffective 
instruction  due  to  the  general  lack  of  expertise  in  computer- 
based  instructional  systems  (Spector,  1990) . 

The  Advanced  Instructional  Design  Advisor  project  is  divided 
into  four  phases: 


Phase  1: 
Phase  2 : 
Phase  3 : 
Phase  4 : 


Conceptualization  &  Functional  Specifications 
Conceptual  Refinement  &  System  Specifications 
Prototype,  Field  Test,  &  Refinement 
Technology  Demonstration  &  System  Validation 


The  first  two  phases  have  been  performed  by  Task  Order 
Contracts.  The  third  phase  is  being  accomplished  via  a  Broad 
Agency  Announcement  (BAA) .  The  fourth  phase  will  be  completed  by 
a  fully  specified  contract.  The  work  reported  herein  concerns 
the  first  phase. 

The  two  papers  that  comprise  this  report  focus  on  problems 
of  particular  difficulty  that  may  be  encountered  in  the  process 
of  designing  and  developing  a  system  to  advise  and  assist 
inexperienced  instructional  technologists  in  the  task  of  building 
effective  CBI.  Friedman  addresses  the  difficulties  in  advising 
CBI  developers  about  the  effective  use  of  graphics.  Poison 
addresses  a  range  of  issues  raised  in  the  context  of  CBI 
development  efforts  at  an  Air  Force  Technical  Training  Center. 
Both  offer  a  sobering  view  of  the  task  confronting  the 
development  of  an  AIDA,  but  each  considers  the  task  possible  and 
worth  pursuing. 

In  section  II  Friedman  argues  that  we  know  less  about  the 
effective  use  of  graphics  than  is  necessary  in  order  to  build  a 
graphics  advisor  for  an  AIDA.  The  common  assumption  is  that  more 
graphics  is  better.  Uittle  regard  is  given  to  the  particular 
nature  of  a  graphic  given  a  specific  teaching  objective  and 
context.  She  illustrates  this  point  nicely  by  collecting  a 
series  of  very  different  DNA  diagrams  all  of  which  can  be  used 
effectively  in  a  particular  context.  Given  the  complexity  of  the 
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task  of  advising  a  novice  how  to  make  effective  use  of  graphics 

in  CBI,  she  then  provides  a  framework  or  architecture  for  an  AIDA 

graphics  advisor. 

Poison  also  comments  on  the  graphics  problem  in  CBI.  Her 

sources  indicate  that  the  single  most  time-consuming  aspect  of 

developing  CBI  concerns  the  development  of  supporting  graphics. 
She  reiterates  how  inadequate  many  of  the  resulting  graphics  are. 
Poison  goes  on  to  emphasize  that  designing  a  graphics  mini¬ 
advisor  is  no  trivial  task  and  is  about  as  complicated  as  the 
task  of  advising  CBI  developers  about  instructional  strategies. 

In  addition,  Poison  identifies  several  additional  problem  areas, 
including  CBI  evaluation,  interactivity,  task  and  instructional 
analysis,  and  the  human-computer  interface.  Because  each  of 
these  areas  causes  particular  challenges  for  the  design  of 
effective  CBI,  additional  research  should  be  conducted  in  each 
area  as  AIDA  is  designed  and  developed.  For  example,  having 
advised  a  user  about  an  appropriate  design  for  a  particular 
problem  is  not  sufficient.  The  advice  must  be  presented  in  such 
a  way  that  it  is  understandable,  meaningful,  and  executable. 

While  the  next  two  sections  pose  challenging  problems  that 
must  be  resolved  before  success  with  AIDA  can  be  expected,  they 
should  not  be  read  as  posing  insurmountable  difficulties. 

Rather,  they  should  be  viewed  as  research  agendas  to  be  conducted 
along  with  the  AIDA  project. 
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II.  DESIGNING  GRAPHICS  FOR  COURSEWARE  (Friedman) 


Background 

This  research  was  supported  in  part  by  a  grant  from  the 
Natural  Sciences  and  Engineering  Research  Council  of  Canada.  I 
would  like  to  thank  Paul  Hearty  for  some  very  thoughtful  and 
Insightful  discussions  about  the  area  in  general,  and  Martha 
Poison  for  her  helpful  comments  on  the  manuscript. 

Designers  of  instructional  systems  (automated  or  otherwise) , 
as  well  as  designers  of  courseware,  often  assume  that  graphic 
materials  and  media  almost  always  play  a  useful  and  even 
necessary  role  in  the  development  of  educational  materials,  and, 
by  inference,  in  the  acquisition  of  knowledge  and  skills. 

However,  in  reviewing  the  educational,  psychological,  and  human 
factors  literatures,  there  is  scant  evidence  to  support  this 
view.  This  chapter  surveys  these  three  literatures,  and  focuses 
on  data  relevant  to  understanding  how  and  when  graphic  and  other 
non-text  representations  can  support  the  acquisition  of  factual 
knowledge  and  procedural  skills.  It  then  goes  on  to  describe 
aspects  of  the  human  information  processing  architecture  that 
need  to  be  taken  into  account  in  the  selection  of  any  media  of 
instruction;  this  part  of  the  chapter  is  based  upon  principles  of 
cognitive  science.  Third,  a  framework  for  the  use  of  graphics  in 
the  development  of  courseware  is  presented,  using  data  surveyed 
as  well  as  the  architectural  considerations.  Several  of  the 
modules  within  this  framework  could  be  implemented  in  an 
automated  instructional  design  system  as  "mini-experts.” 
Suggestions  for  research  that  would  provide  the  necessary 
information  to  implement  these  experts  are  proposed  throughout 
this  chapter. 


Introduction 


Graphics  and  other  forms  of  non-text  representations 
undoubtedly  play  an  important  role  in  the  acquisition  of  factual 
knowledge  and  procedural  skills.  It  goes  without  saying,  then, 
that  guidelines  for  the  design  of  courseware,  technical  manuals, 
and  other  instructional  materials  should  include  prescriptions 
for  the  use  of  graphics  that  maximize  the  efficiency  with  which 
individuals  learn  the  material  at  hand.  These  guidelines  should 
prescribe,  among  other  things,  (a)  when  using  a  graphic  is 
preferable  to  using  text,  (b)  what  sort  of  graphic  representation 
(realistic;  schematic;  etc.)  is  best  suited  to  the  particular 
educational  applications  and  goals,  (c)  what  sort  of  graphical 
conventions  (e.g.,  that  we  read  figures  from  left  to  right)  may 
be  assumed  to  be  known  by  the  targeted  student  population,  and 
(d)  when  it  is  desirable  for  a  graphic  to  be  redundant  with  text 
information  and  when  it  is  desirable  for  the  graphic  to 
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supplement  the  text.  Some  of  these  prescriptions  will  be 
specific  to  a  particular  content  area  (e.g.,  in  teaching 
chemistry,  some  notational  schemes  are  more  useful  than  others) , 
whereas  others  will  generalize  across  content  areas. 

In  developing  an  automated  instructional  design  system,  such 
as  AIDA,  the  principles  underlying  the  role  of  graphics  in 
knowledge  and  skill  acquisition  roust  be  incorporated  into  the 
system.  The  system  should  allow  subject  matter  experts,  who  may 
not  necessarily  be  aware  of  the  educational  and  psychological 
issues  underlying  the  role  of  graphics  in  learning,  to  make  the 
most  efficacious  use  of  graphics  in  the  courseware  they  develop. 
In  addition,  a  system  for  automating  the  selection  of  graphics 
for  courseware  must  be  designed  to  be  able  to  dovetail  with  those 
parts  of  the  system  in  which  the  course  modules  themselves  are 
developed.  In  the  theoretical  framework  outlined  in  the  second 
section  of  this  chapter,  several  of  the  components  (e.g.,  the 
information  parser;  the  representation  analyzer)  could  be 
developed  as  "mini-experts”  and  incorporated  into  a  larger 
system.  These  "mini-experts"  can  be  construed  as  "graphics 
technologists"  for  the  system  as  a  whole.  However,  as  will  be 
seen,  there  exist  sufficient  gaps  in  our  knowledge  about  the  role 
of  graphics  in  the  learning  process  that  much  research  is 
required  before  such  "mini-experts"  will  be  able  to  inform  the 
courseware  development  process.  Some  suggestions  for  the  areas 
in  which  research  is  required  are  given  throughout  the 
discussion. 

That  educators  believe  graphics  are  important  for  learning 
is  evident  from  the  fact  that  pictures,  graphs,  diagrams, 
symbols,  and  other  non-text  information  is  found  in  all  levels  of 
instructional  materials,  from  first-grade  readers  to  senior-  and 
graduate- level  university  course  materials;  the  inclusion  of 
graphics  in  these  materials  is  obviously  intended  to  facilitate 
learning.  Pictorial  representations  are  also  ubiquitous  in 
training  manuals,  consumer  products,  assembly  instructions,  and 
the  like;  businesses  that  devise  or  use  such  materials  operate 
under  the  implicit  or  explicit  assumption  that  it  is  easier  to 
understand,  follow,  or  :  emember  information  that  is  presented 
this  way.  In  addition,  more  and  more  information  in  the  media 
available  to  the  lay  public  is  being  presented  in  graphic  form, 
and  is  meant  to  be  combined  with  nonredundant  information  from 
text,  the  assumption  being  that  the  intended  information  and  the 
appropriate  inferences  are  acquired  as  a  matter  of  course. 

Despite  the  widespread  use  of  non-text  representations, 
however,  the  psychological  literature  about  how  adults  acquire 
knowledge  and  make  inferences  from  graphics  is  sparse,  as  are 
data  about  how  people  integrate  nonredundant  text  and  non-text 
information  and  whether  their  ability  to  do  so  interacts  with 
their  level  of  literacy  or  their  expertise  within  a  specialized 
domain.  For  example,  comparisons  of  novice  and  expert  physics 
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problem-solvers  indicate  that  the  use  of  graphics  to  present 
problem  information  to  novices  might  be  contraindicated  because 
the  graphics  draw  attention  to  superficial  problem  features 
(Larkin,  McDermott,  Simon  &  Simon,  1980) .  Yet,  most  guidelines 
for  constructing  non-text  learning  materials  are  based  on  the 
largely  intuitive  (and  uncritical)  assumptions  that  (a)  pictures 
are  good,  (b)  more  pictures  are  better,  and  (c)  realistic 
pictures  are  best  of  all  (e.g.,  Dwyer,  1972;  see  Friedman,  1986, 
Holliday,  1973,  and  Moore  &  Nawrocki,  1978,  for  review). 

There  is  also  a  lack  of  research  about  which  aspects  of 
non-text  representations  best  convey  different  types  of 
information;  little  is  known  about  the  kinds  of  graphical 
conventions  implicit  in  many  representations  (e.g.,  that  we  read 
diagrams  from  top  to  bottom  and  left  to  right;  that  size  can 
convey  relative  quantity;  that  physical  contiguity  often  implies 
temporal  contiguity) ;  nor  is  there  much  information  about  how 
these  conventions  are  acquired  or  best  employed  in  the 
construction  of  learning  materials.  Related  to  this  issue  is  the 
fact  that  there  are  few  empirically-based  guidelines  about  how 
best  to  convey  different  substantive  categories  of  information 
using  graphics.  For  example,  to  convey  that  DNA  is  structured 
like  a  double  helix,  with  two  sets  of  complementary  base  pairs 
that  separate  to  form  templates  for  daughter  strands,  one  must 
convey  information  about  appearances,  states  and  state  changes, 
structures,  functions,  processes,  etc.  To  do  this  successfully 
requires  a  principled  means  of  choosing  one  or  another  portrayal 
method  as  being  better  suited  to  a  particular  application.  This, 
in  turn,  will  likely  require  shifting  the  research  emphasis  from 
concerns  about  the  specific  physical  characteristics  of  a 
particular  graphic  media  (e.g.,  color  or  monochrome;  drawings  vs. 
slides;  large  vs.  small  format)  to  the  type  of  information  that 
needs  to  be  conveyed  and  the  known  (or  to  be  learned)  conventions 
available  to  portray  that  type  of  information. 

In  the  first  section  of  this  chapter,  I  survey  the 
psychological,  educational,  and  human  factors  literatures 
relevant  to  understanding  how  non-text  representations  can 
support  the  acquisition  of  factual  knowledge  and  procedural 
skills.  In  surveying  these  literatures,  several  things  became 
dismayingly  apparent.  First,  for  the  most  part,  these  are  three 
distinct  literatures  with  different  (though  occasionally 
overlapping)  sets  of  emphases,  methods,  empirical  constraints, 
applied  concerns,  subject  populations,  theoretical  objectives, 
and  so  on.  These  and  other  differences  render  cross-literature 
comparisons  difficult.  Second,  common  to  all  three  literatures 
is  an  adherence  to  the  almost  uncritical  assumption  that  graphics 
are  better,  whether  for  memory  and  learning  (psychology) , 
conceptual  understanding  and  motivation  (education; ,  or  ergonomic 
considerations  (human  factors).  Moore  and  Nawrocki  (1978) 
discuss  several  reasons  often  cited  for  an  adherence  to  this 
assumption,  including  the  beliefs  that  pictures  are  effective 
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because  (a)  they  are  easier  to  perceive  than  verbal  stimuli,  (b) 
they  can  be  realistic  (the  assumption  here  being  that  learning 
should  be  somehow  proportional  to  degree  of  "pictorial 
fidelity"),  (c)  pictures  can  decrease  memory  load  (or  increase 
"channel  capacity")  by  providing  redundant  codes,  (d)  individual 
differences  in  spatial  ability  are  important,  so  some  students 
will  profit  from  pictorial  materials  and  others  will  not,  and  (e) 
students  are  more  motivated  to  learn  from  pictorial  materials. 

In  the  present  review,  it  will  become  apparent  that  almost 
all  of  these  reasons  are  too  broadly  stated  and  all  can  be 
questioned  on  empirical  grounds.  For  example,  although  iconic  or 
pictorial  materials  might  produce  better  speed  and  accuracy  than 
verbal  materials  for  some  tasks  (e.g.,  perceptual  vs.  memorial 
comparisons,  Moyer  &  Bayer,  1976;  speeded  inference^:,  Friedman  & 
Bourne,  1976),  there  are  clearly  other  tasks  (e.g.,  naming  words 
vs.  objects.  Potter  &  Falconer,  1975  [see  Snodgrass,  1980])  for 
which  verbal  stimuli  produce  superior  performance.  Indeed,  in 
picture  naming  tasks,  variables  such  as  the  frequency  in  print  of 
the  pictures'  names  have  been  shown  to  influence  performance 
(Oldfield  &  Wingfield,  1965) .  Similarly,  although  individual 
differences  in  spatial  ability  should  be  important  a  priori  in 
some  educational  contexts  (e.g.,  solid  geometry),  they  certainly 
might  not  be  in  others,  including  contexts  in  which  pictures  have 
been  shown  to  facilitate  performance  (e.g.,  learning  how  to 
identify  and  classify  dinosaurs,  Winn,  1982) .  The  main 
objectives  of  this  chapter  are  as  follows: 

(a)  to  determine  the  general  conditions  under  which 
graphics  are  an  effective  educational  tool, 

(b)  to  specify  which  types  of  gr hies  (e.g.,  line 
drawings,  grey-scale  images,  photographs,  etc.) 
facilitate  learning  in  particular  knowledge  and  skill 
domains,  and 

(c)  to  develop  a  theoretical  framework  that  provides 
guidelines  for  the  use  of  graphics  in  the  development 
of  courseware  that  incorporates  the  information  from 
the  first  two  objectives. 


Limitations  of  Current  Research 

All  three  of  the  literatures  reviewed  have  limitations.  In 
the  psychological  literature,  the  main  emphasis  has  been  on  how 
graphical  information  is  perceived  and  remembered,  rather  than  on 
how  different  types  of  graphics  convey  different  types  of 
information  more  or  less  efficiently,  or  how  graphics  can  be  used 
to  best  represent  information  about  a  particular  subject  domain, 
or  indeed,  what  information  should  be  represented  to  achieve 
certain  educational  goals.  Thus,  the  main  limitation  on  the 


usefulness  of  this  literature  can  be  characterized  by  the  fact 
that  people  investigate,  for  example,  how,  whether,  or  why 
picture  memory  is  better  than  verbal  memory,  but  not  how, 
whether,  or  why  pictures  should  be  used  in  the  service  of 
learning  a  particular  topic  or  type  of  information. 

A  second  limitation  in  the  psychological  literature  is  that 
the  current  emphasis  is  on  trying  to  characterize  knowledge  and 
process  differences  between  novices  and  experts  in  a  variety  of 
domains,  and  not  necessarily  on  how  expertise  is  best  acquired. 
Thus,  the  direct  educational  implications  of  such  work  are  often 
missing.  Finally,  th^  research  emphasis  has  been  almost  entirely 
placed  on  the  psychological  differences  that  may  exist  between 
pictorial  representations  and  other  representations,  such  as 
text.  Comparisons  among  different  types  of  graphic 
representations  (e.g.,  photos  vs.  line  drawings)  are  relatively 
rare. 


Research  in  education  has  also  focused  rather  heavily  on 
picture-text  differences  and  physical  variables  (e.g.,  large  vs. 
small  displays) ,  to  the  virtual  exclusion  of  such  psychologically 
relevant  factors  as  differences  in  the  type  of  information  to  be 
learned,  the  interaction  between  acquisition  and  testing  media, 
how  to  determine  equivalencies  in  information  content  across 
media,  etc.  In  addition,  variations  in  the  subject  matter 
tested,  the  learners'  characteristics,  the  stimulus  variables 
manipulated,  the  type  of  test  given,  the  educational  objectives 
of  the  research,  and  many  other  factors  make  it  exceptionally 
difficult  to  make  comparisons  across  studies  in  this  literature, 
or,  indeed,  to  generalize  at  all. 

As  might  be  anticipated,  in  the  human  factors  literature  the 
emphasis  has  been  on  the  principally  ergonomic  implications  of 
using  different  display  media.  In  addition,  much  of  the 
literature  is  focused  on  extremely  specific  applications. 

Perhaps  most  dismaying  is  the  tendency  to  make  recommendations  in 
the  absence  of  empirical  evidence.  Again,  this  reflects  an 
inherent  assumption  that  the  more  graphics  there  are,  the  more 
ergonomically  sound  will  be  the  application.  There  are  a  few 
enlightening  exceptions  here,  though,  which  will  be  taken  up  in 
turn. 


Despite  these  caveats,  each  of  these  literatures  has 
contributed  to  understanding  the  role  that  graphical  information 
plays  in  acquiring  new  knowledge  and  skills.  Each  has  also 
contributed  to  the  development  of  the  theoretical  framework  to  be 
presented.  In  the  section  that  follows,  I  discuss  constraints 
imposed  by  the  human  information  processing  system  architecture 
and  the  implications  these  have  for  acquiring  information  from 
graphic  and  other  media.  In  particular,  I  discuss  the 
implications  of  (a)  working  memory  (WM)  capacity  limits,  (b)  the 
manner  in  which  information  is  organized  and  represented  in 
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long-term  memory  (LTM) ,  and  (c)  the  potential  existence  of 
qualitatively  different  types  of  processing  resources.  Within 
each  section,  a  few  selected  studies  in  which  learning  from 
different  types  of  graphic  media  was  specifically  investigated 
will  be  discussed,  where  possible.  The  section  will  close  with  a 
section  on  theoretical  and  methodological  issues  that  preclude 
making  strong  generalizations  at  this  point,  as  well  as  with 
recommendations  for  future  research. 


System  Architecture 

Views  regarding  the  architecture  of  the  human  information 
processing  system  have  changed  considerably  over  the  last  two 
decades.  Nevertheless,  there  are  three  enduring  aspects  of  the 
system  that  are  relevant  to  the  role  of  graphics  in  education: 
Limits  on  WM  capacity  have  implications  for  the  use  of  graphics 
as  organizational  aids  for  memory  and  problem  solving,  and  also 
bears  on  issues  of  stimulus  complexity  and  the  highlighting  of 
information  through  the  use  of  color  or  other  means.  The 
hierarchical  organization  of  LTM  has  implications  for  the 
perception  and  comprehension  of  graphic  displays,  and  LTM 
representational  differences  between  text  and  graphics  have 
implications  for  the  attainment  of  expertise  through  the  use  of 
analogy  and  metaphor.  Finally,  the  existence  of  different  types 
of  processing  resources  has  implications  for  the  relative 
efficacy  of  providing  information  presented  in  different  media, 
or  to  different  modalities. 

Working  memory.  Chase  and  Simon  (1973)  showed  that  chess 
experts  can  remember  about  the  same  number  of  randomly  placed 
chess  pieces  as  can  novices,  illustrating  that  the  source  of 
expertise  does  not  rest  with  differences  between  novices  and 
experts  in  some  innate  or  acquired  WM  capacity.  Rather,  at  least 
one  source  of  expertise  lies  in  the  fact  that  experts  have  a  vast 
amount  of  information  stored  in  LTM  about  legal  configurations  of 
chess  pieces;  when  they  view  a  board  that  is  structured  legally 
with  respect  to  the  rules  of  chess,  they  can  "parse”  it  into 
chunks  based  on  these  configurations.  They  then  merely  need  to 
retrieve  the  configurations,  and  generate  the  pieces  that 
comprise  them,  at  recall.  The  novice,  or  course,  has  no  recourse 
to  this  strategy.  These  findings  have  now  been  replicated  in 
many  domains  of  expertise,  ranging  from  other  types  of  games 
(e.g.,  GO;  Reitman,  1976)  to  more  "real-world"  situations,  such 
as  recall  of  maps  depicting  tactical  battlefield  situations 
represented  by  graphical  symbols  (e.g.,  Badre,  1982). 

There  are  several  implications  that  the  relationship  between 
WM  capacity  and  expertise  has  for  the  effectiveness  of  graphics 
in  problem-solving  and  learning.  To  the  extent  that  a  graphic 
representation  of  a  problem  facilitates  chunking,  it  should 
lessen  the  burden  on  WM  and  facilitate  problem-solving  in  domains 
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for  which  WM  limitations  directly  constrain  solution  speed  or 
accuracy  (i.e.,  most  real-world  situations).  For  example, 
Schwartz  and  Fattaleh  (1972)  found  that  when  subjects  were  given 
deductive  reasoning  problems  in  two-dimensional  arrays,  they 
solved  the  problems  faster  than  when  the  problems  were  presented 
in  prose  form. 

A  second  implication  of  the  relationship  between  WM  capacity 
and  expertise  has  to  do  with  information  acquisition  and 
decision-making.  It  is  not  sufficient  merely  to  present 
to-be-learned  information  in  arbitrary  chunks;  it  is  necessary 
that  the  chunks  be  meaningful.  For  example.  Bower  and  Springston 
(1970)  gave  subjects  auditory  lists  of  letters  to  repeat  back; 
identical  sequences  of  letters  were  chunked  into  either  arbitrary 
(e.g.,  IB  MPH  DFB  IX)  or  meaningful  (e.g.,  IBM  PHD  FBI  X) 
groupings.  Although  there  were  the  same  sequence  and  number  of 
letters  and  the  same  number  of  chunks  in  each  case,  the 
meaningful  groupings  produced  much  better  performance. 

Moreover,  there  is  evidence  that  units  of  information  are 
best  presented  in  a  familiar  sequence.  For  example,  Badre  (1982) 
tested  the  recall  of  subjects  who  were  experts  in  tactical 
decision-making  by  presenting  either  meaningfully  chunked 
information  in  a  meaningful  sequence,  or  in  the  reverse  of  the 
meaningful  sequence,  or  nonmeaningful  chunks  presented  in  an 
arbitrary  sequence,  or  all  of  the  information  at  once.  He  found 
that  the  correct  sequence  was  recalled  better  than  the  same 
sequence  in  reverse  order,  which  was  no  different  than  recall  of 
nonmeaningful  chunks.  Importantly,  recall  in  the  group  who 
received  the  correct  sequence  did  not  differ  from  that  of  the 
group  who  received  all  the  information  on  the  same  screen.  Thus, 
if  graphics  can  be  used  as  aids  to  chunking  information,  then 
they  may  prove  additionally  useful  if  the  chunked  information  can 
be  presented  in  meaningful,  familiar  sequences. 

These  examples  illustrate  the  importance  of  taking  the  level 
of  expertise  of  the  subjects  as  well  as  the  semantics  of  the 
stimulus  displays  into  account  when  investigating  manipulations 
that  allegedly  make  a  display  more  comprehensible.  Many 
investigations  of  physical  variables,  such  as  color  coding  and 
stimulus  complexity,  fail  to  do  this.  For  example,  Knapp,  Moses 
and  Gellman  (1982)  investigated  the  effect  of  display  complexity 
on  comprehension.  They  recognized  that  complex  displays  might 
prove  burdensome  because  of  WM  limitations,  among  other  things, 
and  recommended  several  guidelines  for  highlighting  information 
to  make  displays  easier  to  comprehend.  One  such  suggestion  was 
that  a  complex  display  should  be  segmented  to  show  one  segment  at 
a  time.  However,  Knapp  et  al.  (1982)  suggested  segmenting  the 
displays  using  a  grid;  it  is  likely  that  a  grid  might  cross  an 
expert's  chunk  boundary  and  actually  interfere  with 
comprehension.  A  better  approach  would  be  to  segment  the  display 
into  chunks  that  were  based  upon  units  of  information  that  were 
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meaningful  according  to  some  criteria  of  expertise. 


All  three  literatures  under  review  suggest  that  under  many 
circumstances,  "less  is  better,"  which  has  implications  for  the 
role  of  complexity  in  graphical  and  mixed-mode  media 
presentations.  For  example,  Borg  and  Schuller  (1979)  showed  that 
subjects  who  were  learning  the  names  and  locations  of  parts  of  a 
relatively  complex  object  did  better  when  the  inner  details  of 
those  parts  were  omitted  from  the  stimulus  than  they  did  with  a 
photograph  of  the  actual  object.  Similarly,  Dwyer  (1972)  found 
that  either  simple  line  drawings  or  oral  instructions  alone  were 
more  effective  than  realistic  graphics  (e.g.,  shaded  drawings  and 
photographs)  on  tests  of  drawing,  identification,  terminology, 
and  comprehension  in  the  domain  of  heart  physiology  (his  results 
are  quite  a  bit  more  complex  than  this  and  should  be  examined  in 
detail) .  Other  investigators  have  found  that  less  complex 
stimulus  materials  lead  to  equal  or  better  learning  than  more 
complex  versions  of  the  "same"  subject  matter  (e.g.,  Moore, 
Nawrocki  &  Simutis,  1979) ,  and  at  least  some  of  these  findings 
are  likely  to  be  reflecting  WM  capacity  limitations.  There  are 
also  studies  showing  that  less  complex  stimuli  are  easier  to 
identify  from  brief  presentations  than  are  more  complex  stimuli 
(e.g.,  Ryan  &  Schwartz,  1956).  Thus,  it  may  be  that  for  many 
applications  in  which  designers  have  striven  for  increased 
veracity  of  detail  in  their  graphics,  exactly  the  opposite 
approach  is  warranted. 

On  the  other  hand,  there  certainly  may  be  some  circumstances 
(e.g.,  learning  a  difficult  discrimination)  in  which  a  complex 
display  is  not  only  warranted,  but  is  necessary.  An  example  of 
such  a  circumstance  may  be  found  in  the  work  of  Marcel  and 
Barnard  (1979) .  They  showed  subjects  pictographic  sequences  of 
various  actions  and  the  states  that  would  result  from  those 
actions,  using  an  experimental  apparatus  that  actually  paralleled 
the  actions  and  states  of  a  pay  telephone.  The  pictured 
instructions  could  show  either  the  part  of  the  apparatus  that  was 
relevant  to  each  particular  step  in  the  sequence,  or  else  it 
could  show  the  entire  (more  complex)  apparatus  at  each  step. 
Subjects  who  received  only  relevant  part  information  produced 
poorer  verbal  descriptions  of  what  the  instructions  meant  and 
also  performed  the  task  more  poorly  than  subjects  who  received 
pictures  of  the  entire  apparatus.  One  possible  explanation  for 
this  finding  is  that  Marcel  and  Barnard  (1979)  did  not  parse 
their  graphics  properly,  so  that  their  "part"  information  did  not 
include  information  necessary  to  place  the  parts  in  context.  A 
more  interesting  possibility  is  that  Marcel  and  Barnard's 
subjects  were  novices  trying  to  learn  a  procedural  sequence, 
whereas,  for  example,  Badre's  (1982)  tacticians,  whose 
performance  did  not  suffer  from  their  having  received  partial 
information  sequentially,  were  experts  trying  to  integrate 
declarative  information  over  time. 
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Related  to  the  issue  of  stimulus  complexity  is  the  issue  of 
whether  using  color  in  displays  can  contribute  to  their 
perceptibility  or  comprehensibility  (Dwyer,  1975;  El-Gazzar, 

1984;  Lamberski  &  Dwyer,  1983;  Luder  &  Barber,  1984;  Reid  & 
Miller,  1980;  Stone,  1983) .  For  example,  Reid  and  Miller  (1980) 
had  children  write  descriptions  of  the  subject  matter  of  either 
monochrome  or  colored  photographs  of  biological  subjects.  They 
found  both  positive  and  negative  effects  of  color,  and  both  were 
due  to  what  they  believe  is  the  ability  of  color  to  act  as  a 
distractor.  The  positive  effects  were  that  color  displays 
yielded  less  of  a  tendency  to  merely  name  (identify)  objects  or 
their  parts,  which  has  been  interpreted  as  a  less  optimal  outcome 
than  an  actual  description.  The  main  disadvantage  of  the 
addition  of  color  to  the  stimuli  was  that  it  tended  to  distract 
subjects  towards  describing  features  that  were  less  biologically 
significant.  It  should  be  noted,  however,  that  the  photographs 
used  by  Reid  and  Miller  (1980)  were  of  the  objects  in  their 
"natural”  state;  that  is,  they  represented  appearance  information 
rather  than  assisting  chunking.  In  principle,  color  coding  might 
be  used  to  highlight  parts  of  displays  that  are  significant,  thus 
possibly  reducing  the  amount  of  attention  or  capacity  that  might 
be  required  to  extract  this  information  from  a  display. 

An  example  of  this  principle  can  be  found  in  the  results  of 
Luder  and  Barber  (1984) ,  who  investigated  the  effectiveness  of 
redundant  color  coding  in  search  vs.  identification  tasks. 
Redundant  color  coding  refers  to  the  situation  in  which  color  is 
perfectly  correlated  with  some  other  physical  cue  (e.g.,  shape). 
Luder  and  Barber  (1984)  had  subjects  perform  a  continuous 
compensatory  tracking  task  while  periodically  making  judgments 
about  the  state  of  valves  in  a  fuel  system.  Identification 
judgments  involved  questions  like  "valves  2  and  6  are  closed,” 
whereas  search  judgments  involved  questions  like  "there  are  three 
valves  open."  The  valves  could  be  in  three  states  (open  =  green, 
closed  =  blue,  emergency  =  red) .  It  should  be  noted  that  these 
color  choices  are  nonarbitrary.  It  should  also  be  noted  that 
identification  of  state  information  (e.g.,  open)  is  a  subset  of 
the  search  task. 

Luder  and  Barber  (1984)  found  that  identification  was  faster 
than  search  in  the  monochrome  conditions,  whereas  the  reverse  was 
true  in  the  color  conditions.  Essentially,  since  the  locations 
of  the  valves  were  fixed,  redundant  color  coding  offered  no 
advantage  over  either  shape  or  location  for  purposes  of 
identifying  the  valves  or  their  states;  however,  color  did 
provide  substantial  gains  when  subjects  had  to  search  for 
information  across  the  entire  display.  This  is  an  illustration 
of  the  principle  that  the  effectiveness  of  a  particular  type  of 
graphic  variable  will  be  highly  sensitive  to  the  task  demands. 

In  addition,  the  color  group  performed  the  tracking  task  more 
accurately  than  the  monochrome  group  in  both  search  and 
identification  conditions.  This  could  be  interpreted  as  meaning 
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that  redundant  color  coding  enabled  some  capacity  (or  resources) 
to  be  freed  for  use  on  the  tracking  task. 

In  the  framework  developed  below,  general  questions  such  as 
"when  is  it  better  (for  learning  or  comprehension)  to  use  color 
graphics  as  opposed  to  black  and  white  graphics?"  are  seen  as 
typically  inappropriate.  Instead,  it  will  be  necessary  to 
determine,  for  example,  what  the  WM  requirements  of  a  given  task 
environment  are,  whether  the  information  to  be  learned  lends 
itself  to  chunking,  and  if  so,  what  are  the  appropriate  chunks  to 
use  for  the  targeted  student  population.  It  is  only  at  this 
juncture  that  it  makes  sense  to  develop  instructional  materials, 
via  color  coding  or  other  means  (e.g.,  highlighting),  to  try  to 
determine  which  of  several  methods  facilitates  chunking  of 
displays  in  learners  at  various  levels  of  expertise.  The 
knowledge  gained  by  this  sort  of  research  could  be  incorporated 
into  a  "mini-expert"  that  would  enable  courseware  developers  to 
use  graphics  appropriate  to  how  the  material  to  be  learned  was  to 
be  chunked. 

To  summarize,  WM  capacity  limitations  imply  that  the 
efficiency  of  any  media  for  both  learning  and  problem  solving 
will  vary  as  a  function  of  how  well-suited  or  sensitive  that 
media  is  for  transmitting  information  in  chunks  or 
configurations.  For  instance,  unretouched  photographs  require 
the  viewer  to  perform  all  of  the  chunking  operations,  whereas 
photos  with  irrelevant  areas  masked  out,  or  with  lines  drawn 
around  what  is  to  be  considered  a  chunk  alleviate  this  operation. 
Conversely,  any  media  representation  that  obscures  or  otherwise 
precludes  such  chunking  should  interfere  with  efficient  learning 
and  problem  solving.  It  should  be  noted,  however,  that  in  this 
approach,  to  test  relative  efficiency  for  performance  across  or 
within  media  requires  knowing  how  the  information  to  be 
transmitted  should  be  parsed  into  chunks  for  any  given  domain  and 
achieved  or  desired  level  of  expertise.  There  is  a  notable 
absence  of  research  on  this  issue,  although  the  methodology  for 
determining  how  an  individual  has  chunked  a  particular  display  is 
reasonably  well-established  (see  Badre,  1982,  and  Chase  & 

Simon,  1973  for  examples) . 

Long-term  memory.  Research  on  the  perception, 
comprehension,  and  memory  of  pictorial  material  has  been 
intimately  related  to  issues  concerning  the  representation  of 
knowledge.  Although  no  single  theory  of  LTM  representation  has 
been  accepted  as  clearly  preferable  to  its  competitors,  some 
general  principles  have  emerged.  Those  relevant  to  the  role  of 
graphics  in  education  include  the  fact  that  knowledge  in  LTM  is 
organized,  that  the  organization  is  hierarchical,  and  that  there 
are  different  types  of  knowledge  represented  in  LTM. 

At  least  three  types  of  LTM  knowledge  are  potentially 
relevant  to  studying  the  role  of  graphic  and  pictorial 
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information  in  acquiring  new  factual  knowledge  and  procedural 
skills  :  General  world  knowledge  (e.g.,  Friedman,  1979), 
domain-specific  knowledge  (e.g.,  Hegarty,  Just,  &  Morrison, 

1988) ,  and  knowledge  about  graphic  conventions  that  are  either 
domain-specific  or  not  (e.g.,  Winn,  1982).  I  will  discuss  each 
of  these  in  turn. 

Most  memory  theorists  have  agreed  that  there  are  abstract 
structures  within  LTM  that  represent  everyday  knowledge  about  the 
real  world.  The  term  "schema”  is  often  used  to  refer  to  these 
knowledge  structures.  A  schema  is  a  data  structure  that 
represents  those  properties,  objects,  actions,  events,  roles,  and 
so  on,  that  are  most  commonly  encountered  in  a  particular 
instantiation  of  the  schema.  The  schema  for  an  object,  for 
example,  might  represent  typically  encountered  visual  properties 
and  relations  among  parts,  or  it  might  "point  to"  procedures  that 
can  be  used  to  detect  such  properties  and  relations.  An  object 
schema  might  also  represent  what  the  object  does,  where  we  are 
likely  to  find  it,  and  how  we  can  interact  with  it. 

Thus,  there  are  constraints  on  schema  variables  that  define 
the  range  of  values  it  is  most  likely  to  have.  We  know  about 
likelihoods,  ranges,  and  distributions  of  properties,  objects, 
and  events;  indeed,  this  is  typically  what  is  referred  to  as 
world  knowledge.  This  knowledge  is  structured  hierarchically, 
and  plays  an  important  role  in  perception,  comprehension,  and 
memory  of  both  linguistic  and  pictorial  information. 

Both  the  organizational  properties  of  LTM  and  the  default 
knowledge  represented  therein  will  influence  the  acquisition  of 
new  material.  It  has  been  known  for  a  long  time  that 
comprehension  and  memory  are  both  facilitated  when  new 
information  can  be  readily  organized  and  assimilated  into  extant 
knowledge  structures,  but  the  primary  evidence  for  this  has  come 
from  studies  in  which  information  is  to  be  acquired  from  text 
(e.g.,  Haviland  &  Clark,  1974).  That  this  assertion  is  true 
about  information  presented  graphically  can  be  illustrated  in 
several  ways.  At  the  perceptual  level,  Biederman,  Glass,  and 
Stacy  (1973)  had  subjects  identify  target  objects  in  briefly 
presented  scenes  that  were  either  coherent  or  jumbled  .  This 
manipulation  preserved  the  amount  of  "physical"  information  in 
the  pictures  (e.g.,  contours,  brightness  changes,  etc.)  while 
destroying  the  semantic  relationships  among  the  objects.  This 
means  that  a  high-level  schema  could  not  be  used  to  aid 
identification  of  the  objects.  In  one  study,  Biederman  et  al. 
(1973)  found  that  even  when  subjects  knew  what  to  look  for  (i.e., 
they  had  been  shown  the  piece  with  the  target  object  in  advance 
of  the  slide)  as  well  as  where  to  look,  they  were  still  more 
accurate  with  the  coherent  displays  than  with  the  jumbled 
displays. 

In  a  similar  vein,  Mandler  and  Johnson  (1976)  have  shown 
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that  subjects  remember  more  about  black  and  white  line  drawings 
that  are  coherently  arranged  (according  to  some  schema)  than  they 
do  when  given  drawings  of  the  identical  objects  arranged 
haphazardly.  Their  results  are  especially  noteworthy  because 
their  drawings  contained  relatively  few  objects  (e.g. ,  6-8)  that 
were  not  too  detailed,  and  according  to  some  accounts  of  visual 
LTM  (e.g.,  Shepard,  1967)  subjects  might  have  been  expected  to 
remember  them  rather  well,  regardless  of  how  they  were  arranged. 

A  final  example  of  how  organizing  new  information  makes  it 
more  comprehensible  and  memorable  comes  from  the  work  of 
Bransford  and  Johnson  (1973) .  They  showed  that  prose  passages 
that  were  virtually  impossible  to  comprehend,  let  alone  remember, 
could  be  rendered  easy  and  memorable  through  the  addition  of 
advance  information  that  supplied  some  necessary  contextual  and 
organizational  support  (see  Mayer,  1979,  for  a  review  of  the  role 
of  advance  organizers  in  learning) .  They  were  able  to 
demonstrate  these  effects  by  providing  a  title  for  some  of  the 
passages,  and  for  others,  by  providing  a  picture  that  "set  the 
scene"  for  the  actions  that  were  described  in  the  passage 
(unfortunately,  they  never  compared  the  two  methods  directly) . 

In  summary,  the  more  that  a  graphical  display  can  be  made  to 
correspond  to  or  take  advantage  of  the  way  in  which  information 
is  organized  in  LTM,  the  better  will  the  information  in  that 
display  be  apprehended  and  remembered. 

In  addition  to  the  role  played  by  the  organization  of  LTM, 
there  are  three  broad  classes  of  LTM  expectations,  or  world 
knowledge,  that  are  relevant  to  the  role  of  graphics  in 
education.  The  first,  of  course,  is  the  "everyday"  type  of 
knowledge  just  discussed.  Expectations  about  the  world  govern 
what  people  look  at  in  a  picture  (Loftus,  1972) ,  how  long  it 
takes  them  to  recognize  what  they  see  (Friedman,  1979) ,  and  the 
duration  of  subsequent  fixations  to  the  same  objects  (Friedman  & 
Liebelt,  1981) .  More  generally,  when  schemas  guide  perception 
(as  they  do  in  most  real  world  situations) ,  then  objects  in  the 
environment  that  correspond  to  "slots"  in  an  activated  schema 
could  be  perceived  and  comprehended  relatively  automatically  . 

In  contrast,  without  context,  or  in  an  unusual  context,  object 
identification  usually  requires  more  visual  details  (Friedman, 
1979;  Palmer,  1975). 

One  implication  of  this  approach  is  that  people  should  take 
less  time  to  identify  expected  objects  than  unexpected  objects. 
This  should  hold  for  objects  in  the  environment  as  well  as 
objects  depicted  graphically.  This  conjecture  has  been  supported 
by  eye  fixation  data  recorded  from  subjects  who  viewed  shaded 
line  drawings  depicting  common  scenes  and  places.  First 
fixations  to  expected  objects  were  half  as  long  as  first 
fixations  to  unexpected  objects  (Friedman,  1979) .  Since  all 
objects  had  roughly  equivalent  amounts  of  detail,  subjects  either 
processed  such  details  more  quickly  when  identifying  expected 
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objects,  or,  more  likely,  such  details  were  unnecessary  for 
identifying  expected  objects  in  context.  The  latter 
interpretation  was  supported  by  recognition  memory  data. 

Subjects  virtually  never  noticed  changes  made  to  the  details  of 
expected  objects  whereas  they  often  noticed  changes  of  details  to 
the  unexpected  objects.  Indeed,  changes  of  details  to  unexpected 
objects  were  noticed  more  often  than  when  expected  objects  were 
deleted  altogether. 

These  findings  indicate  that  the  expected  portion  of  a 
stimulus  might  be  stored  as  an  instance  of  the  particular  global 
schema  it  instantiates,  without  regard  to  specific  episodic 
(occurrence)  or  descriptive  details,  since  it  is  not  normally 
useful  to  take  note  of  such  already  expected  information.  Thus, 
in  an  educational  context,  it  may  be  unnecessary,  or  even 
detrimental  in  certain  circumstances,  to  present  graphical 
stimuli  that  are  rich  in  detail,  especially  when  that  detail  is 
only  for  the  purpose  of  embellishment. 

A  case  in  point  can  be  found  in  the  education  literature 
about  what  is  learned  from  a  text  that  either  is  or  is  not 
accompanied  by  illustrations.  Some  authors  find  that  graphics 
facilitate  written  or  oral  text  comprehension  and  memory  (e.g., 
Holliday,  1975;  Lesgold,  Levin,  Shimron,  &  Guttman,  1975; 
Pressley,  Levin,  Pigotte,  LeCompte  &  Hope,  1983;  Rigney  &  Lutz, 
1976;  Royer  &  Cable,  1976;  Ruch  &  Levin,  1977)  and  others  find 
that  they  do  not  (Alesandrini  &  Rigney,  1981;  Edyburn,  1982; 

King,  1975;  Lang  &  Soloman,  1979;  Rohwer  &  Harris,  1975).  Haring 
and  Fry  (1979)  claimed  that  previous  studies  were  in  disagreement 
because  the  subjects  were  different  ages,  the  picture 
manipulations  ranged  from  one  relevant  picture  per  passage  that 
either  was  specific  or  general  to  37  pictures  per  passage,  the 
texts  varied  in  difficulty  and  type  (e.g.,  whether  they  were 
narrative  or  expository) ,  and  the  measures  of  comprehension 
varied  from  multiple  choice  to  free  recall.  Using  fourth  and 
sixth-grade  subjects,  they  demonstrated  that  additional  details 
in  a  picture  do  not  facilitate  text  comprehension  if  those 
details  are  at  a  "low  level."  In  the  present  context,  the 
details  they  refer  to  are  those  that  embellish  expected  items. 

The  second  class  of  expectations  that  are  relevant  to  the 
role  of  graphics  in  education  is  knowledge  about  the  substantive 
domain  to  be  learned  or  about  domains  that  are  analogous  to  it. 

We  have  already  seen,  for  example,  how  the  amount  of  information 
that  is  acquired  from  a  display  is  related  to  the  amount  of 
information  that  can  be  chunked  and  held  in  WM.  This  in  turn,  is 
a  positive  function  of  the  amount  of  prior  knowledge  that  the 
observer  has  about  the  domain  being  displayed  and  how  that 
information  is  organized. 

Educators  take  advantage  of  this  principle  by  exploiting 
analogies  between  an  already  known  domain  and  a  to-be-learned 
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domain.  Of  interest  for  the  role  of  graphics  in  these  efforts  is 
a  study  by  Royer  and  Cable  (1976).  They  were  investigating  the 
role  of  transfer  in  learning  from  prose  passages  and  were  trying 
to  establish  whether  illustrations  and  physical  analogies  could 
facilitate  transfer.  They  conjectured  that  transfer  between  two 
passages  would  require  that  the  first  passage  somehow  be  able  to 
establish  a  "knowledge  bridge"  to  the  second.  They  had 
college-age  subjects  read  an  abstract  passage  about  the  internal 
structure  of  metals  and  electroconductivity  after  reading  either 
an  irrelevant  (control)  passage,  another  abstract  passage  about 
electroconductivity,  a  passage  about  the  topic  that  used  concrete 
physical  referents,  a  passage  that  was  abstract  but  that  used 
analogies  to  known  concepts,  or  the  same  abstract  passage 
accompanied  by  line  drawings  showing  the  structural  relationships 
being  referred  to.  The  latter  three  manipulations  all  improved 
performance  relative  to  the  control  group  and  to  the  abstract 
passage,  and  they  did  not  differ  from  each  other.  Thus,  it  was 
the  concreteness  per  se  that  facilitated  transfer  —  whether  by 
physical  referent,  by  analogy,  or  by  illustration.  This  study  is 
important  because  it  illustrates  that  it  is  sometimes  not  the  use 
of  graphics  per  se,  but  rather,  a  property  of  graphics  (such  as 
concreteness)  shared  by  other  media  that  has  benefitted 
performance.  The  exact  componential  aspect  of  a  graphic 
manipulation  that  has  helped  (or  hindered)  performance  has  been 
rarely  researched  or  discussed. 

The  third  class  of  LTM  knowledge  that  is  relevant  to  the 
communicative  effectiveness  of  graphics  in  education  is  knowledge 
about  graphical  conventions  (e.g.,  the  use  of  lines  streaming 
away  from  a  figure  to  indicate  motion;  the  assumption  that  we 
will  read  a  table  from  left  to  right,  top  to  bottom;  the 
assi  i ption  that  occlusion  will  be  interpreted  as  a  depth  cue  in  a 
line  drawing,  etc.).  That  is,  in  the  text  comprehension 
literature  a  distinction  has  been  made  between  the  knowledge 
needed  to  comprehend  the  events  that  occur  in  a  story  and  the 
knowledge  we  have  about  how  stories  are  typically  structured 
(e.g.,  that  a  story  consists  of  a  setting  and  episodes;  that 
episodes  consist  of  conflicts  and  resolutions,  etc.;  Mandler  & 
Johnson,  1977) .  A  similar  distinction  can  be  made  for  graphical 
stimuli:  a  graphic  will  normally  attempt  to  convey  some 
substantive  information,  and  it  will  do  so  using  a  "grammar"  and 
"syntax"  which  are  assumed  to  be  shared  between  the  designer  of 
the  graphic  and  the  targeted  observer.  This  is  clearly  an  area 
that  lends  itself  to  automatization  in  the  development  of 
courseware;  in  the  theoretical  framework  proposed  below,  for 
example,  it  is  the  task  of  the  information  parser  to  determine, 
for  a  given  course  module,  which  graphical  conventions  can  be 
assumed  to  be  known,  and  which  are  novel.  When  novel  conventions 
are  required,  the  system  might  suggest  to  the  courseware 
developer  that  a  submodule  be  created. 

Although  there  are  certainly  substantive  domains  in  which 
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information  is  principally  graphic  and  in  which  it  is  obvious 
that  there  are  conventions  that  need  to  be  learned  before 
comprehension  is  easy  or  even  possible  (e.g.,  map  reading, 

Potash,  1977;  radiography,  Carmody,  1985;  aerial  photography. 

Way,  1973,  cited  in  Perkins,  1980),  for  most  of  the  domains  we 
are  concerned  with  here,  it  is  not  even  clear  whether  graphics 
are  an  appropriate  means  to  convey  information,  let  alone  which 
graphical  conventions  might  or  might  not  be  appropriate  for  doing 
so. 


There  has  not  been  much  systematic  research  conducted  on  the 
relative  comprehensibility  of  various  graphical  conventions  for 
conveying  different  kinds  of  information.  Evidence  that  there 
are  such  conventions,  and  that  certain  experiences  (or  training) 
might  be  necessary  to  perceive  accurately  things  that  are 
normally  encountered  in  three  dimensions  when  they  are  pictured 
in  two  dimensions  comes  from  the  cross-cultural  literature  on 
picture  perception  (see  Pick  &  Pick,  1978,  for  a  review). 

Although  interpreting  cultural  differences  in  picture  perception 
is  difficult,  in  the  present  context,  the  etiology  of  such 
differences  is  far  less  important  than  the  fact  that  they  exist 
at  all.  Their  existence  underscores  the  importance  of  not 
presuming  that  the  conventions  implicit  in  a  particular  method  of 
depicting  information  will  be  known  to  the  audience  for  which 
that  graphic  is  intended. 

An  almost  poignant,  though  revealing,  example  of  the 
importance  of  this  principle  comes  from  a  study  by  Lang  and 
Soloman  (1979),  who  investigated  whether  pictures  would 
facilitate  the  process  of  learning  to  read  common  nouns.  In  one 
study,  only  children  who  had  been  told  that  the  pictures  were 
representations  of  the  objects  named  by  the  words  showed  improved 
performance.  Thus,  at  least  for  young  children,  it  cannot  even 
be  assumed  that  they  understand  the  conventional  use  of  proximity 
(and  simultaneity  in  time)  to  indicate  that  two  things  should  be 
taken  as  having  the  same  referent. 

In  another  example  of  the  role  that  learned  graphic 
conventions  play  in  acquiring  new  knowledge,  Winn  (1982) 
investigated  several  means  by  which  diagrams  could  be  structured 
to  convey  different  types  of  information.  His  grade  nine 
subjects  were  to  learn  to  identify  dinosaurs  as  well  as  to  learn 
their  correct  sequence  of  evolution.  All  subjects  received  flow 
diagrams  with  dinosaur  names;  half  the  subjects  saw  pictures  of 
the  dinosaurs  above  their  names.  In  addition,  for  half  of  each 
of  these  groups,  the  flow  diagram  presented  the  dinosaurs  in  the 
"canonical”  (conventional)  left-to-right,  top-to-bottom  order, 
whereas  the  other  half  received  the  flow  diagrams  in  the  reverse 
of  this  "expected"  sequence.  Winn  (1982)  found  that  the 
conventional  order  produced  better  performance  on  the  test  of 
evolutionary  sequence  than  the  reverse  order,  but  only  for 
subjects  who  had  received  pictures  in  addition  to  names.  This 
result  is  reminiscent  of  the  Badre  (1982)  findings  described 
earlier.  Both  findings  are  especially  interesting  because  they 
illustrate  how  entrenched  a  convention  can  become;  in  principle. 
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a  reverse  order  has  identical  sequence  information  as  a 
non-reversed  order,  yet  they  are  clearly  psychologically 
different. 

That  graphic  conventions  are  not  automatically  comprehended, 
and  that  such  comprehension  is  necessary  for  certain  types  of 
learning  is  nicely  demonstrated  in  an  experiment  by  Brooks 
(1977) .  She  showed  second,  sixth,  and  ninth  graders  18  black  and 
white  line  drawings  that  each  contained  two  normally  inanimate 
objects,  one  of  which  was  drawn  with  arms  and  legs.  The  objects 
were  shown  engaged  in  interactive  relationships,  and  half  the 
subjects  saw  the  pictures  with  "action  lines,"  such  as  vertical 
lines  drawn  above  an  object  to  indicate  that  it  is  falling.  It 
should  be  noted  that,  though  action  lines  are  not  entirely 
arbitrary  (for  example,  vertical  lines  are  not  used  to  indicate 
horizontal  or  diagonal  motion) ,  they  are  nevertheless  relatively 
arbitrary  conventions  used  to  depict  the  direction  and  amplitude 
of  motion.  Brooks'  (1977)  rationale  for  the  manipulation  was 
that  action  lines  could  be  used  as  clues  to  the  interactions 
between  objects  if,  and  only  if,  subjects  understand  this 
particular  convention.  Thus,  older  children,  who  presumably  had 
more  experience  with  comics  and  cartoons,  would  benefit  from  the 
action  lines  more  than  younger  children.  She  found  that  action 
lines  only  facilitated  recall  for  the  ninth  grade  subjects. 

Action  lines  represent  an  example  of  a  pictorial  convention 
that  is  acquired  relatively  late,  so  that  their  use  as  a 
mechanism  to  improve  the  "readability"  of  a  graphic  needs  to  be 
constrained  by  this  fact.  Indeed,  several  conclusions  that  may 
be  reached  from  this  literature  are  that  (a)  different  pictorial 
devices  or  conventions  need  to  be  explicitly  identified, 

(b)  different  pictorial  conventions  need  to  be  scaled  along  a 
dimension  of  "readability,"  (c)  this  scale  may  differ  according 
to  the  skills  and  experience  of  the  observers  (Perkins,  1980) , 
and,  (d)  the  conventions  that  might  be  particular  to  a  given 
subject  matter  domain  might  need  to  be  explicitly  taught,  much  as 
a  mathematical  notation  is  taught  prior  to  (or  at  least 
simultaneously  with)  its  use  in  a  proof.  Each  of  these 
conclusions  could  easily  serve  as  a  focus  of  future  research. 

To  summarize,  the  organization  of  LTM  and  its  representation 
of  world  knowledge,  special  domain  knowledge,  and  knowledge  of 
graphic  conventions  will  clearly  have  a  profound  influence  on 
perceptibility,  comprehensibility  and  memorability,  both  inside 
and  outside  of  a  formal  educational  context.  Thus,  media 
representations  that  exploit  this  organization,  that  take 
advantage  of  conventions  and  default  knowledge  to  make  analogies 
between  what  is  known  and  what  is  to  become  known  should 
facilitate  learning. 

Processing  resources.  Until  1979,  most  accounts  of  the 
architecture  of  the  human  information  processing  system  assumed 
that  the  processes  that  took  place  within  the  system  all  drew  on 
the  same  general  pool  of  processing  resources  (e.g.,  Kahneman, 
1973;  Norman  &  Bobrow,  1975).  Then,  Navon  and  Gopher  (1979) 
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wrote  a  seminal  article  in  which  they  proposed  that  the  human 
information  processing  system  probably  has  access  to  several 
qualitatively  different  types  of  resources,  each  of  which  was 
limited  in  amount.  Processes  that  required  the  same  type  of 
resource  to  execute,  when  performed  together,  would  have  to 
compete  for  that  particular  resource,  with  the  possibility  that 
there  would  not  be  enough  for  all  competitors  and  hence  there 
might  be  a  decline  in  performance  relative  to  a  noncompetitive 
situation.  In  contrast,  processes  that  required  qualitatively 
different  resources  might  be  able  to  be  executed  concurrently 
with  no  loss  of  either  efficiency  or  accuracy. 

Navon  and  Gopher  (1979)  did  not  specify  what  the  qualitative 
properties  of  different  resource  pools  might  be,  and  indeed, 
their  model  is  somewhat  intractable  because  it  has  no  mechanism 
for  the  a  priori  specification  of  which  types  of  resources  will 
be  required  to  perform  a  particular  task.  In  part  as  an  effort 
to  address  this  problem,  both  Friedman  and  Poison  (1981)  and 
Wickens  (1984)  have  proposed  multiple  resource  models  in  which 
the  nature  and  number  of  resource  pools  have  been  specified.  In 
Friedman  and  Poison's  (1981)  model,  two  independent  resource 
pools  are  assumed  to  exist  that  are  each  associated  with  a 
particular  cerebral  hemisphere.  In  Wicken's  (1984)  model, 
separate  resource  pools  are  hypothesized  to  exist  for  information 
input  to  different  modalities  (e.g.,  auditory  vs.  visual),  for 
different  types  of  stimulus  codes  (e.g.,  verbal  vs. 
visuospatial) ,  and  for  different  stages  of  processing  (early  vs. 
late)  . 


These  approaches  are  relevant  to  the  role  of  graphics  in 
education  because  two  longstanding  hypotheses  in  the  education 
literature  regarding  why  the  addition  of  graphics  to  a  text  or 
aural  presentation  should  facilitate  information  acquisition  are 
that  (a)  learning  is  facilitated  when  stimuli  are  input  to 
different  "channels,"  which  usually  is  interpreted  to  mean 
different  modalities,  and  (b)  learning  is  facilitated  when, 
either  because  of  input  to  different  modalities  or  because 
stimuli  are  of  qualitatively  different  types  (e.g.,  pictures  and 
text) ,  their  processing  results  in  more  than  one  code.  Both 
hypotheses  are  typically  tested  by  augmenting  a  verbal  passage 
(presented  either  aurally  or  visually)  with  some  sort  of  graphic 
stimuli  (e.g.,  Nugent,  1982;  Pressley,  Levin,  Pigott,  LeCompte,  & 
Hope,  1983;  Rohwer  &  Harris,  1975).  From  a  multiple-resources 
view,  processing  stimuli  in  different  modalities  or  of  different 
types  might  indeed  imply  that  different  types  of  resources  are 
necessary.  The  question  is  whether  this  is  beneficial  or  not. 

Just  as  the  issue  of  redundancy  of  codes  has  been  relevant 
to  the  literature  concerned  with  the  relative  merits  of  color  vs. 
monochrome  displays,  the  costs  and  benefits  of  redundancy  is  an 
issue  in  the  "multimedia"  literature.  It  should  be  noted  that 
there  can  be  several  types  of  redundancy,  which  are  often 
confused.  In  the  "pure"  cases,  there  can  be  redundancy  of  input 
channel,  as  when  pictures  and  text  are  presented  visually,  or 
redundancy  of  code,  as  when  the  same  text  is  presented  aurally 
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and  visually.  There  can  also  be  hybrids,  as  when  aurally 
presented  verbal  material  is  supplemented  by  pictures.  An 
example  of  the  latter  is  a  study  by  Pressley,  et  al.  (1983). 

They  read  aloud  lists  of  concrete  sentences  to  second  and  third 
graders.  The  sentences  were  either  presented  by  themselves,  or 
with  pictures  that  matched  their  content.  They  found  that  the 
matching  condition  produced  better  recall  than  either  of  the 
other  conditions,  which  did  not  differ  from  each  other.  Thus, 
Pressley  et  al.  (1983)  confirmed  that  pictures  that  are 
redundant  with  the  semantic  content  of  a  sentence  facilitate 
memory  for  that  content.  However,  they  had  no  real  control  over 
the  degree  of  mismatch;  indeed,  some  of  the  mismatched  pictures 
directly  contradicted  the  sentences  they  were  shown  with.  This 
illustrates  an  important  issue  in  this  literature:  How  is  one  to 
know,  either  across  or  even  within-media,  what  meaning  is 
afforded  by  each  instance  of  a  given  concept? 

Rohwer  and  Harris  (1975)  also  investigated  the  effects  of 
presenting  information  in  different  modalities  and  media,  and 
their  study  is  notable  because  they  used  single-media  control 
groups.  They  presented  high  and  low  socioeconomic  status  (SES) 
fourth-graders  with  three  expository  prose  passages.  In  the 
single-medium  versions,  the  passages  were  either  presented  with  a 
tape  recorder,  or  printed  versions  were  presented  via  slides  of 
the  text  of  the  passage,  or  picture  versions  were  presented  that 
consisted  of  10  pictures  per  passage.  In  multimedia 
presentations,  all  possible  combinations  of  media  pairs  were 
presented,  as  well  as  a  condition  in  which  the  information  was 
presented  in  all  three  media  simultaneously.  It  should  be  noted 
that  each  passage  contrasted  two  related  concepts  (e.g.,  two 
types  of  monkeys)  on  each  of  five  attributes  (e.g,,  type  of 
tail) ,  so  that  the  correspondence  between  the  semantic  content  of 
the  passage  and  its  pictures  was  made  more  feasible. 

Although  the  results  of  their  study  were  complex,  the  main 
findings  were  that,  for  single-media,  either  oral  or  printed 
presentations  produced  better  performance  that  the  picture 
presentation.  For  combined  media,  although  the  outcome  depended 
a  bit  on  the  SES  of  the  subjects,  both  the  oral  plus  picture  and 
the  print  plus  picture  presentations  produced  better  performance 
than  the  oral  plus  print  conditions.  Rohwer  and  Harris  (1975) 
concluded  that  presenting  the  same  semantic  content  in  two 
different  codes  (or  what  we  have  been  referring  to  as  "code 
redundancy”)  is  generally  better  than  presenting  the  identical 
(verbal)  information  to  two  different  modalities. 

Nugent  (1982)  came  to  a  similar  conclusion,  although  from 
slightly  different  findings.  She  pointed  out  that  a  medium  is 
seldom  associated  with  only  one  symbol  system.  For  example,  a 
visual  medium,  such  as  film,  can  be  used  to  display  still  or 
animated  graphics  in  addition  to  text.  She  conjectured  that  when 
semantic  information  is  redundant,  then  presenting  it  tr 
different  modalities  via  different  symbol  systems  (i.e., 
presenting  prose  to  the  auditory  system  and  pictures  visually) 
should  maximize  learning.  Using  a  design  similar  to  that  of 
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Rohwer  and  Harris  (1975) ,  she  found  positive  evidence  for  this 
conjecture.  In  a  second  study,  Nugent  (1982)  found  that  when 
semantic  information  was  nonredundant ,  then  presenting  it  to 
different  modalities  did  not  augment  learning,  relative  to  a 
single-media  control,  but  neither  did  it  interfere  with  learning. 
That  is,  subjects  who  received  different  information  orally  and 
visually  performed  as  well  as  subjects  who  received  only  the  oral 
or  only  the  visual  information. 

There  has  not  been  enough  research  in  this  area  to  come  to 
any  firm  conclusions  regarding  the  role  of  either  redundancy  of 
modality  or  code  in  efficient  learning.  A  major  problem  may  be 
that  it  is  difficult  to  specify  exactly  what  information  is  being 
conveyed  by  a  particular  media.  This  problem  will  be  taken  up 
below. 


Theoretical  and  Methodological  Issues 

There  are  several  methodological  problems  that  plague  the 
literature  as  a  whole  and  that  require  resolution  before 
questions  regarding  the  role  of  graphics  in  education  can  be 
answered.  Indeed,  some  of  these  problems  preclude  making  almost 
any  generalizations  at  all  from  extant  data. 

One  particularly  vexing  problem,  mentioned  above,  is  that  of 
stimulus  equivalence;  it  is  difficult  to  determine  exactly  what 
information  is  being  conveyed  by  different  stimulus  materials. 
This  is  particularly  so  when  comparisons  are  made  between 
different  classes  of  representations  (e.g. ,  comparisons  of  text 
with  flowcharts  or  movies  or  a  series  of  pictures) ,  but  it  is 
also  difficult  when  comparisons  are  made  between  two  different 
types  of  graphic  stimuli  that  are  alleged  to  portray  identical 
substantive  information  in  merely  different  formats.  What  is 
more,  very  few  investigators  make  an  effort  to  substantiate 
claims  of  equivalence  by  subjecting  their  materials  to  norming 
studies,  for  example. 

The  problem  of  stimulus  equivalence  between  media  has  been 
studied  in  its  own  right  (Baggett,  1979,  1986;  Baggett  & 
Ehrenfeucht,  1982) .  Baggett  and  Ehrenfeucht  (1982)  point  out 
that  there  has  basically  been  no  method  of  preparing 
information  for  experimentation  which  is  the  same  in  content  but 
which  can  be  presented  in  two  media  (e.g.,  either  pictorially  or 
verbally) .  Thus,  similarities  and  differences  in  performance  as 
a  function  of  media  of  presentation  may  be  due  to  differences  in 
content,  and/or  differences  in  the  way  the  two  media  convey 
messages;  they  certainly  may  not  be  unequivocally  assigned  to 
media  differences. 

Baggett  (1979)  devised  a  method  for  constructing  a  text  that 
was  structurally  equivalent  to  a  wordless  movie  (The  Red 
Balloon) ,  insofar  as  subjects  could  agree  that  both  the  movie  and 
the  constructed  text  contained  the  same  14  episodes,  each  with  an 
exposition,  complication,  and  resolution,  and  that  episodic 
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boundaries  in  one  medium  had  exactly  specified  locations  in  the 
other.  New  subjects  then  either  watched  the  movie  or  heard  the 
text,  and  then  recalled  as  much  as  they  could.  Recall  of 
structural  statements  was  similar  in  both  media,  but  there  were 
differences  in  content  recall,  which  were  attributed  to  different 
degrees  to  which  different  world  knowledge  was  activated  in  the 
two  groups. 

In  a  second  series  of  studies,  Baggett  and  Ehrenfeucht 
(1982)  specifically  tried  to  equate  content  as  well  as  structure 
between  a  narrative  movie  fThe  Unicorn  in  the  Garden)  and  a  text. 
They  developed  four  measures  of  empirical] y-determined  content 
equivalence:  (a)  ratings  of  how  well -represented  each  of  350 

sentence  fragments  was  in  the  movie  (or  vice  versa) ,  (b)  for  each 
fragment,  which  photo  (taken  from  the  movie)  best  corresponded  to 
it  (or  vice  versa);  termed  "bi-directional  touchpoints, "  (c) 
which  of  the  350  sentence  fragments  or  photos  were  in  the  top  20% 
in  terms  of  importance  to  the  story,  and  (d)  the  similarity  of 
importance  ratings  of  the  characters  in  the  two  medias.  It 
should  be  noted  that  these  methods  allow  assessment  of  degree  of 
equivalence,  and  are  not  limited  to  comparisons  between  movie  and 
text  media  (e.g. ,  two  texts  or  two  films  could  be  compared) .  New 
subjects  recalled  the  stories,  and  the  summaries  based  on 
different  media  were  indistinguishable. 

Baggett's  work  is  notable  because  the  development  of 
materials  was  not  based  on  intuition  (as  it  usually  is  in  this 
literature)  but  rather,  it  was  based  on  several  empirical 
measures.  Although  it  is  not  clear  whether  her  conclusions  will 
hold  in  a  context  in  which  non-narrative  information  is  to  be 
learned,  nor  is  it  clear  whether  her  methods  could  be  adapted  to 
other  graphic  media,  it  would  be  a  great  help  in  interpreting 
research  if  other  investigators  would  take  as  much  care  to 
determine  in  what  ways  their  materials  were  or  were  not 
equivalent.  Otherwise,  any  between-  or  within-media  differences 
obtained  are  at  best  ambiguous.  Thus,  one  possibility  for  future 
research  is  the  development  of  methods  and  measures  for  assessing 
stimulus  equivalence. 

The  second  methodological  issue  shall  be  referred  to  as 
domain  specificity,  and  concerns  the  choice  of  subject  matter  to 
be  learned.  It  is  almost  overwhelmingly  true  that  either 
between-  or  within-media  comparisons  are  made  in  one  and  only  one 
subject-matter  domain,  and  often  on  only  one  particular  topic  or 
lesson  within  that  domain.  So,  for  example,  Lamberski  and  Dwyer 
(1983)  contrasted  color  vs.  monochrome  presentations  of  the 
physiology  of  the  heart;  Holliday,  Brunner,  and  Donais  (1973) 
compared  picture-word  vs.  block -word  diagrams  of  the  oxygen, 
carbon,  nitrogen  and  water  cycles;  Nugent  (1982)  compared  the 
efficacy  of  print,  audio  and  movie  media  (and  their  combinations) 
on  the  topic  of  cheetahs;  Rohwer  and  Harris  (1975)  compared  text 
alone  with  text  plus  analogies  or  pictures  for  learning  about 
electroconductivity,  and  so  on.  The  point  is  that  there  needs  to 
be  much  more  attention  paid  to  content  domains  in  comparing 
different  media  presentations.  It  would  also  be  useful  to 
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compare  more  than  one  topic  or  lesson  within  the  same  domain,  so 
that  proper  item  analyses  could  be  conducted.  The  ability  to 
generalize  findings  either  within  or  across  substantive  domains 
will  be  severely  constrained  unless  specific  empirical  work  is 
conducted  to  compare  these  domains.  Thus,  here  is  another  area 
that  is  ripe  for  future  research. 

Third,  there  is  a  notable  lack  of  research  on  the  issues  of 
(a)  what  graphical  conventions  exist  or  are  assumed  to  exist  in 
the  presentation  of  information  from  any  particular  substantive 
domain,  (b)  whether  these  conventions  can  be  taught  and  if  it  is 
helpful  to  do  so,  (c)  what  conventions  are  being  implicitly 
compared  with  each  other  in  any  given  study,  and  (d)  how  graphic 
conventions  can  themselves  be  exploited  to  facilitate  information 
acquisition.  These  issues  have  already  been  discussed  (e.g.,  see 
Winn,  1981,  for  an  explicit  comparison) ;  they  become 
methodological  issues  for  reasons  similar  to  those  for  issues  of 
stimulus  equivalence.  That  is,  if  two  types  of  graphics  are 
being  compared  and  some  sort  of  behavioral  difference  is 
observed,  one  needs  to  determine  whether  at  least  one  of  the 
reasons  is  that  one  type  of  graphic  exploits  or  enlists  a 
particular  convention  (known  or  unknown  to  the  subjects)  whereas 
the  other  does  not.  Once  again,  unless  this  is  known  and/or 
controlled,  interpretations  of  differences  between  conditions 
cannot  be  made  unambiguously. 

A  fourth  area  of  concern  has  to  do  with  the  encoding-test 
relationship;  i.e.,  the  relationship  between  how  information  is 
presented  and  what  sort  of  knowledge  is  being  tested.  There  are 
several  psychological  principles  (e.g.,  encoding  specificity)  as 
well  as  several  studies  (e.g.,  McDaniel,  Friedman,  &  Bourne, 

1978;  Stein,  Morris,  &  Bransford,  1978)  indicating  that  unless 
there  is  compatibility  between  the  way  materials  are  encoded  and 
how  they  are  tested,  degree  of  learning  (indeed,  even  what  is 
learned)  may  be  incorrectly  estimated.  For  example,  Lamberski  and 
Dwyer  (1983)  gave  subjects  color  or  monochrome  pictures  of  the 
human  heart,  and  tested  them  on  terminology,  comprehension  of 
heart  function,  identification  of  structures,  and  their  ability 
to  draw  the  heart.  They  found  more  of  a  difference  between 
presentation  conditions  on  tests  requiring  greater  visualization 
(e.g.,  identification  of  structures  and  drawing).  Similarly, 

Jeon  and  Branson  (1981),  who  investigated  whether  movies,  slides, 
or  text  would  be  better  for  acquisition  of  a  motor  skill, 
criticized  previous  between-media  studies  on  the  grounds  that 
many  v/ritten  behavioral  measures  do  not  match  the  instructional 
objectives  (i.e.,  actual  motor  performance).  They  cite  Allen  and 
Weintraub  (1968)  as  suggesting  that  the  "use  of  motion  in  a 
display  is  definitely  indicated  when  the  particular  content  to  be 
learned  consists  of  the  movement  itself  or  its  characteristics, 
or  where  the  content  is  enhanced  or  differentiated  by  the  cues 
provided  in  the  action  of  the  movement.”  Jeon  and  Branson  (1981) 
found  that  no  differences  between  acquisition  conditions  were 
obtained  when  subjects  were  asked  to  write  down  the  procedure 
they  just  learned.  On  the  other  hand,  the  film  produced  much 
better  performance  of  the  actual  task  than  did  either  the  slides 
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or  the  text,  and  with  37%  savings  in  time  to  learn.  Thus,  in 
addition  to  the  media  of  presentation,  it  might  be  profitable  to 
have  routinely  several  types  of  knowledge  tests. 

Fifth,  even  if  investigators  attend  to  the  four  problem 
areas  just  discussed,  there  is  no  guarantee  that  a  conclusion 
regarding  the  relative  merits  of  a  certain  type  of  graphic  can  be 
reached  unless  it  is  clear  that  the  information  being  presented 
has  been  "packaged”  suitably  with  respect  to  the  particular 
domain  of  expertise  to  be  learned  and  the  level  of  knowledge  of 
the  learner.  This  is  the  problem  of  domain  parsing.  Thus, 
research  on  the  role  of  graphics  in  education  cannot  really  take 
place  outside  the  context  of  curriculum  research  in  general. 


Conclusions 


Each  of  the  issues  raised  above  converges  on  a  principle 
conclusion:  Given  the  current  state-of-the-art,  it  is  probably 
premature  for  educators  to  contrast  different  media  and  look  for 
student  aptitude  by  media  of  presentation  interactions. 
Similarly,  it  may  be  insufficient  for  psychologists  to  theorize 
about  differences  between  pictorial  and  verbal  memory  as  if  such 
differences  were  generalizable  across  substantive  domains,  or  for 
human  factors  engineers  to  assert  that  particular  types  of  media 
or  display  technologies  will,  under  all  circumstances,  be  better 
than  others.  This  conclusion  is,  by  no  means,  unique  to  the 
present  article.  For  example,  in  a  1973  review  of  the  education 
literature  on  pictorial  research,  Holliday  stated  that  "It  is 
probable  that  certain  kinds  of  pictures  facilitate  the  learning 
of  certain  types  of  objectives  for  certain  students  with  certain 
characteristics.  However,  the  precise  relationships  have  not 
been  established."  Unfortunately,  the  substance  of  this 
statement  is  still  principally  true. 

As  stated  earlier,  most  studies  investigating  the  role  of 
graphics  in  learning  and  education  have  tested  the  effects  of 
supplementing  verbal  material  (usually  written  text)  with 
representational  (i.e.,  realistic)  pictures.  The  goal  has 
usually  been  to  verify  the  assumption  that  graphics  increase 
instructional  effectiveness.  What  seems  to  be  absent  from  the 
literature  in  general  is  a  systematic  analysis  of  what  types  of 
information  graphical,  as  contrasted  to  some  other 
representations,  can  convey,  and  under  what  circumstances  or  task 
domains  or  sets  of  educational  goals  it  is  necessary  or  desirable 
to  convey  this  information.  Thus,  the  issue  should  probably  not 
be  stated  as:  Under  what  circumstances  and  with  what  sorts  of 
learners  are  graphical  media  an  effective  educational  tool? 
Rather,  there  needs  to  be  more  emphasis  placed  on  analyzing  the 
types  of  concepts  to  be  conveyed  within  a  given  subject  matter 
domain,  and  then  to  investigate  the  circumstances  in  which 
graphical  media  can  be  used  to  convey  these  concepts. 

It  should  be  clear  that,  when  considered  in  terms  of  the 
architectural  hardware  of  the  human  information  processing 
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system,  the  role  played  by  graphics  in  learning  is  really  no 
different,  in  principle,  than  the  role  played  by  text  or  any 
other  media.  Thus,  for  example,  we  know  that  chunking  new 
information,  organizing  it,  giving  it  contextual  support, 
relating  it  to  prior  knowledge,  and  so  on,  are  all  useful  devices 
for  facilitating  knowledge  acquisition.  It  should  come  as  no 
surprise  that  these  principles  are  true  across  media.  What  is 
notably  absent  from  the  literature  is  a  sense  of  the 
circumstances  under  which  a  particular  type  of  media  presentation 
is  better  than  some  other  type  for  chunking  or  organizing  or 
activating  prior  knowledge.  There  are  no  empirical  or 
theoretical  guidelines  for  determining  what  sorts  of  information 
are  uniquely  afforded  by  graphics  and  when  it  is  important  to 
present  such  information.  That  is,  despite  hundreds  of  studies, 
the  question  still  remains  as  to  exactly  what,  if  anything,  is 
special  about  graphics  qua  graphics. 


Theoretical  Framework 


There  is  a  growing  consensus  among  cognitive  scientists  that 
to  perform  successfully  in  a  particular  task  domain,  a  person 
must  have  an  accurate  mental  model  of  that  domain.  Domains  that 
have  been  investigated  in  this  context  include  solving  arithmetic 
or  physics  problems  (Greeno,  1983;  Larkin,  McDermott,  Simon  & 
Simon,  1980) ,  learning  to  use  a  hand-held  calculator  (Mayer  & 
Dayman,  1981;  Young,  1983)  or  a  computer's  text  editor  (Egan  & 
Schwartz,  1979;  Centner  &  Centner,  1983),  and  navigating  around 
large-scale  environments  (Chase  &  Chi,  1981) .  If  accurate  mental 
models  underlie  successful  performance,  then  understanding  the 
role  of  graphics  in  education  requires  determining  what  the 
relevant  mental  models  are  within  particular  knowledge  and  skill 
domains  and  which  type  of  representations  best  convey  them. 

Thus,  to  the  extent  that  mental  models  can  be  conveyed  with 
pictures,  graphics,  diagrams,  and  the  like,  then  optimizing  the 
use  of  graphics  in  education  requires  a  framework  that  describes 
the  relationship  between  different  types  of  graphics  and 
different  educational  goals.  This  will  require  a  careful 
analysis  of  each  knowledge  domain  and  a  more  sophisticated  view 
than  has  been  adopted  in  the  past  toward  the  potential  utility  of 
graphic  representations.  That  is,  there  will  likely  be  some 
domains  for  which  graphics  are  the  method  of  choice  for 
presenting  information  and  others  in  which  their  use  is  even 
contraindicated . 

Teaching  physics  is  a  good  case  in  point.  Physics  problems 
are  often  conveyed  to  students  with  pictorial  diagrams  of  objects 
like  springs,  pulleys,  inclined  planes,  etc.  Yet  this  type  of 
presentation  may  not  be  optimal  because  it  focuses  the  student  on 
superficial  features  (e.g.,  "This  is  an  inclined  plane  problem") 
rather  than  the  physical  laws  that  are  relevant  to  the  problem's 
solution  (Larkin,  McDermott,  Simon  &  Simon,  1980) .  Similarly,  as 
discussed  in  the  previous  section,  there  may  be  certain 
educational  goals  (or  conveyance  needs)  for  which  it  is  better  to 
use  simple  rather  than  complex  graphics,  or  in  which  the  use  of 
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"enhancements”  such  as  color  are  actually  harmful. 

In  the  previous  section,  I  concluded  that  many  issues  need 
to  be  resolved  before  the  role  that  graphic  representations  play 
in  the  educational  process  can  be  fully  understood.  The  five 
particular  issues  raised  were  (a)  stimulus  equivalence;  the 
necessity  of  ensuring  that  two  or  more  media  being  compared 
portray  the  same  information,  or  else,  of  knowing  which 
information  is  unique  to  each,  (b)  graphical  conventions:  the 
need  to  identify  and  make  explicit  the  graphical  conventions  that 
are  employed  either  within  a  particular  experimental  manipulation 
or  as  part  of  the  domain  knowledge  to  be  learned,  (c)  encoding 
specificity:  the  need  to  be  aware  of  the  relationship  between  how 
information  is  presented  and  how  it  is  tested;  that  is,  the  need 
to  acknowledge  the  likelihood  that  there  will  exist  interactions 
between  acquisition  media  and  test  media,  (d)  domain  specificity: 
the  necessity  of  comparing  particular  methods  of  portrayal  (e.g., 
photographs  vs.  schematic  line  drawings)  across  subject  matter 
domains  as  well  as  across  different  topics  within  the  same 
domain,  and  (e)  domain  parsing:  the  need  to  understand  how  a 
particular  domain  of  knowledge  should  be  "parsed"  for  optimal 
presentation  to  learners  of  a  given  level  of  expertise.  The 

"parse"  should  include  a  characterization  of  the  type(s)  of 

knowledge  that  are  to  be  conveyed  as  well  as  suggestions  for  the 

type(s)  of  representations  that  would  best  convey  them. 

I  also  concluded  that  the  role  played  by  graphics  in 
learning  is  in  principle  no  different  than  the  role  played  by  any 
other  information  conveyance  method,  to  the  extent  that  all  of 
the  characteristics,  constraints,  limitations,  and  so  on  imposed 
by  the  information  processing  system  itself  will  be  imposed  on 
all  new  information  indiscriminately.  That  is,  the  processing  of 
all  representations  would  be  constrained  by  the  relevant  parts  of 
the  system  architecture.  Nevertheless,  the  goal  is  to  developed 
a  principled  means  of  choosing  one  or  another  portrayal  method  as 
being  better  suited  to  a  particular  application,  and  to 
incorporate  these  principles  into  an  automated  instructional 
design  system.  To  achieve  the  first  goal  will  require  shifting 
the  research  emphasis  from  how  a  graphic  portrays  something 
(e.g.,  in  color  or  monochrome;  in  detail  or  schematically)  to 
what  needs  to  be  conveyed  in  the  portrayal  (e.g.,  information 
about  structures,  functions,  processes  and  procedures,  rules, 
etc.).  Only  then  might  a  comparison  across  media  types  prove 
fruitful.  That  is,  although  I  do  not  underestimate  the 
importance  of  finding  aptitude  treatment  interactions  and 
understanding  their  etiology,  it  might  do  to  recognize  that  there 
are  other  interactions  -  such  as  that  between  the  type  of 
information  to  be  learned  and  the  type  of  information  best 
afforded  by  a  given  type  of  graphic  representation  -  which  might 
be  more  fundamental  to  this  enterprise. 

In  the  present  section,  I  outline  an  approach  toward  the 
role  of  graphics  in  learning  and  memory  that  tries  to  accommodate 
the  factors,  such  as  domain  specificity  and  graphical 
conventions,  that  were  identified  as  likely  to  be  important  in 
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assessing  the  effectiveness  of  different  methods  of  presenting 
information.  I  first  discuss  three  different  methods  of 
categorizing  graphics.  Two  —  media  of  presentation  and  amount 
of  realism  —  have  been  used  almost  exclusively  (if  implicitly) 
in  past  research,  and  the  third  —  type  of  information  afforded 
—  is  being  advocated  within  the  current  approach.  I  then 
outline  a  conceptual  framework  and  discuss  each  of  its 
components.  I  argue  that  knowledge  domains  should  be 
characterized  according  to  the  type  of  information  needed  to 
acquire  expertise  and  that  these  characterizations  should  be 
formulated  in  terms  of  representational  needs.  Finally,  I  take 
the  reader  through  a  detailed  example,  to  illustrate  the  most 
important  concepts  in  the  framework. 

Categorizing  Graphics  Media  differences  and  the  dimension  of 
realism.  Previous  efforts  to  construct  a  theoretical  framework 
to  describe  the  role  of  graphics  in  education  have  typically 
emphasized  either  physical  differences  between  visual  media  or 
differences  between  types  of  graphic  representations  along  the 
dimension  of  realism.  In  addition,  as  mentioned  earlier,  a  great 
deal  of  research  in  education  combines  these  two  methods  of 
categorization  to  investigate  the  relative  efficiency  of  learning 
from  aural  or  visual  prose  by  itself  versus  learning  when  the 
prose  is  augmented  by  relatively  realistic  graphic 
representations.  Or,  learning  from  prose  is  compared  with 
learning  from  some  other  sort  of  representation,  such  as  a 
flowchart.  With  a  few  specific  exceptions,  this  approach  has  not 
been  a  useful  way  to  proceed. 

Media  differences  refer  to  actual  distinctions  that  exist 
between  different  physical  methods  of  conveying  visual 
information,  such  as  film,  TV,  CRT  pages,  filmstrips,  slides, 
posters,  photographs,  drawings,  etc.  Standard  textbooks  about 
using  audiovisual  media  in  teaching  often  discuss  the  media 
according  to  such  physical  characteristics  (e.g.,  projected 
versus  nonprojected  media;  Erickson  &  Curl,  1972),  or  else, 
according  to  pragmatic  issues  such  as  availability. 

From  the  present  perspective,  it  is  generally  inappropriate 
to  compare  different  physical  media  unless  they  afford  the 
conveyance  of  the  same  type  of  information  or  else,  the  stimulus 
equivalence  Issues  are  at  least  known.  For  instance,  film  and 
video  both  afford  movement  information  directly,  in  real  time. 

In  contrast,  line  drawings  afford  movement  information 
indirectly,  through  the  use  of  conventions  like  action  lines  that 
require  prior  experience  to  interpret.  Moreover,  since  a  single 
drawing  does  not  have  temporality,  a  sequence  of  such  drawings  is 
necessary  to  convey  a  given  movement  from  beginning  to  end,  and 
thus,  not  only  must  information  be  inferred  between  drawings,  but 
each  drawing  must  be  explicitly  recognized  as  part  of  the 
previous  temporal  sequence  and  integrated  within  it.  Thus,  to 
the  extent  that  information  about  movement  is  an  important  part 
of  the  knowledge  to  be  conveyed  within  a  domain,  media 
comparisons  between  film  or  TV  and  more  static  representations 
like  drawings  may  be  most  informative.  However,  this  particular 
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example  is  almost  an  exception.  For  the  most  part,  different 
physical  media  can  convey  the  same  semantic  information  (e.g.,  in 
the  simplest  case,  one  could  present  a  text  in  almost  any  media, 
including  film) .  Thus,  although  media  do  differ  on  physical 
dimensions  like  size  and  resolution,  for  the  most  part  such 
physical  differences  are  probably  unimportant.  This  is  not  to 
ignore  the  fact  that  some  media  may  have  characteristics  that 
make  them  more  desirable  than  others.  For  example,  a  child  may 
be  able  to  control  a  filmstrip  by  him  or  herself,  or  a  large 
format  presentation  may  be  more  "attention-getting”  than  a  small 
format  presentation,  or  it  might  be  less  expensive  or  difficult 
to  make  materials  in  one  media  than  another.  However,  these 
considerations  are  irrelevant  when  considering  the  ability  of  a 
particular  medium  to  convey  a  particular  type  of  information, 
which  is  the  perspective  of  the  present  report. 

On  the  other  hand,  differences  in  the  extent  to  which  a 
visual  representation  maintains  fidelity  to  its  referent  are  at 
least  an  intuitively  plausible  means  by  which  some  types  of 
graphic  representations  might  be  better  than  others  for 
portraying  information  within  particular  learning  environments. 
Table  1  shows  several  different  kinds  of  graphic  representations 
and  where  they  seem  to  be  ordered  along  a  realistic-  abstract 
dimension. 


lype  of  Fidelity 
REALISTIC 

Dtpletivc/Pletorlal 

Schematic 

Iconic 

Structural/Functlonal 

Symbolic 

Arbitrary 

ABSTRACT 


Examples 


3D  Modala,  color  photographa, 
and  ahadad,  datallad  drawings 

Maps,  monochrome  line  drawings, 
and  caricatures 

International  eigne  and 
hieroglyphics 

Flowcharts,  writing  diagrams, 
blueprints,  and  graphs 

Traffic  signs,  logos,  and 
symbols  (e.g.,  aheriffa  badge  or 
skull  and  croaabonea) 

Tables,  charts,  and  text 


Table  1 .  Categorization  of  Graphics 
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It  can  be  seen  that,  generally  speaking,  pictorial  or 
depictive  representations  resemble  their  referents  more  closely 
than  schematic  or  iconic  representations,  in  which  relations  are 
normally  correct  but  the  objects  represented  are  usually  more 
abstract  than  their  referents.  It  should  be  noted,  however,  that 
even  relatively  abstract  representations  such  as  flowcharts  and 
wiring  diagrams  often  maintain  at  least  functional  and  sometimes 
structural  fidelity  to  their  referents.  More  important,  much 
like  physical  media  differences,  the  amount  of  fidelity  that  is 
maintained  by  a  representation  is  likely  to  be  irrelevant  unless 
considered  within  the  context  of  the  purpose  for  which  the 
representation  is  likely  to  be  used.  Thus,  for  example,  in  one 
of  the  most  comprehensive  investigations  of  the  use  of  graphics 
in  science  education,  Dwyer  (1972)  found  that  "small  amounts"  of 
realistic  details  added  to  line  drawings  of  the  heart  were  more 
effective  for  learning  heart  structures  than  were  colored 
photographs  of  an  actual  heart  that  were  presumably  more 
realistic.  In  this  instance,  the  line  drawings  emphasized  the 
structures  to  be  learned,  whereas  the  photographs  were  mostly 
homogeneous . 

Neither  the  medium  of  presentation  nor  the  extent  to  which  a 
representation  is  realistic  can  be  used  exclusively  as  a 
meaningful  basis  for  categorizing  graphic  representations.  This 
is  because  neither  of  these  factors  by  itself  takes  account  of 
differences  in  the  information  conveyance  requirements  that  might 
be  imposed  by  a  particular  subject  matter  domain.  That  is,  just 
as  encoding  specificity  needs  to  be  taken  into  account  when 
evaluating  whether  a  given  test  is  sensitive  to  a  particular 
method  of  acquisition,  researchers  in  graphic  instruction  need  to 
be  more  aware  of  the  potential  influence  of  the  subject  matter 
per  se. 

Information  affordances  and  knowledge  requirements.  One 
potentially  productive  way  to  categorize  different  types  of 
graphic  representations  is  in  terms  of  the  type(s)  of  information 
they  each  afford  (e.g.,  Gibson,  1966).  Having  done  so,  it  would 
then  make  sense  to  compare  different  representations  against  each 
other  in  terms  of,  for  example,  the  directness  or  explicitness  or 
efficiency  with  which  they  represent  and  convey  particular  types 
of  information. 

If  graphics  are  to  be  categorized  in  terms  of  the 
information  they  afford,  it  is  necessary  to  identify  and  possibly 
also  categorize  types  of  information  per  se.  Table  2  shows  a 
partial  classification  of  particular  types  of  information  that 
different  representations  might  convey  more  or  less  well.  The 
types  listed  are  principally  visual  in  nature.  Thus,  for 
example,  characteristics  such  as  weight,  smell,  hardness,  and  the 
like,  are  not  considered. 

It  should  be  emphasized  that  the  categories  in  Table  2  are 
based  on  intuition,  and  that  other  classifications  are  certainly 
plausible,  such  as  one  that  distinguishes  between  static 
information  (e.g.,  objects  and  states)  and  dynamic  information 
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(e.g. ,  processes  and  stages) .  Ultimately,  the  classification  that 
is  most  useful  in  terms  of  describing  information  conveyance 
needs  and  affordances  will  have  to  be  determined  empirically. 

The  table  defines  three  broad  classes  of  information,  and 
two  subcategories  within  each.  The  three  classes  are  information 
about  visual  appearance,  information  about  static  spatial 
relations,  and  information  about  events  and  sequences  of  events. 
All  three  classes  of  information  can  convey  a  concept  or  part  of 
a  concept  either  directly  or  via  analogy  or  metaphor.  For 
example,  if  we  know  that  an  important  structural  characteristic 
of  the  solar  system  is  that  its  planets  revolve  around  the  sun, 
then  by  asserting  that  an  atom  is  like  the  solar  system  we  are 
predicating  that  same  structural  characteristic  to  the  structure 
of  an  atom  (Centner,  1983) .  Thus,  in  principle,  the  same 
representation  could  be  used  for  each  domain. 

The  distinction  between  primary  and  secondary 
characteristics  has  been  made  because  it  might  be  important  or 
useful  to  distinguish  the  information  or  data  that  are  directly 
represented  from  the  inferences  or  conclusions  that  such  data  can 
support.  So,  for  example,  the  primary  characteristics  listed 
under  appearance  information  can,  singly  or  in  combination, 
support  conclusions  about  object  identity  or  category  membership, 
but  the  latter  information  is  not  directly  given  by  the 
appearance  of  an  object.  Similarly,  primary  information  about 
location  supports  computations  of  distance,  and  information  about 
temporal  relationships  supports  conclusions  about  rate  and 
duration  of  change. 

Type  of  Primary  Secondary 

Information  Characteriatice  Characteristics 


Appearancea 


ahapa,  color 
alza,  and  taxturo 


Idantity,  catagory  mambarahlp, 
and  visual  almilarity 


Static  Ralatlona 
Events/Sequences 


orientation, 
spatial  location,  and 
Intamal  structure 

collections  of  objects, 
aasoclativa 
ralatlonshipa, 

cauaa-affact 

relationships, 

temporal 

relationships, 
and  action 
sequences 


distance,  spatial  proximity, 
structural  similarity,  and 

functional  similarity 

temporal  proximity, 
rate  of  change, 
amount  of  change, 
duration  of  states, 
and  duration  of  change 


Tabie2.  Categorization  of  Information  lypes 
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In  general,  once  different  information  types  have  been 
distinguished  and  appropriately  categorized,  it  should  be 
possible  to  compare  graphic  representations  to  determine  how  well 
they  convey  each  type.  It  is  only  at  this  juncture  that 
different  graphical  conveyance  methods  can  be  categorized  in 
terms  of  information  affordances. 

Bobrow  (1975)  pointed  out  that  representations  differ  in  the 
explicitness  with  which  they  afford  information.  Thus,  there  is 
an  important  difference  between  primary  and  secondary  information 
types  and  the  relative  explicitness  with  which  they  are 
represented.  For  example,  the  fact  that  a  square  is  an 
equilateral  polygon  is  only  implicit  in  its  appearance  and  must 
therefore  be  derived  from  a  pictorial  representation  (though  it 
may  be  a  trivial  operation  to  do  so) . 

In  contrast,  a  propositional  (or  text)  representation  of  the 
concept  "square"  could  have  explicit  arguments  stating  that 
squares  have  four  equal  sides  and  angles,  but  then  the  appearance 
of  the  square  would  have  to  be  derived.  Similarly,  a  photograph, 
a  line  drawing,  a  film,  and  a  text  could  all  explicitly  represent 
the  primary  appearance  information  that  canaries  are  yellow, 
whereas,  excluding  the  possibility  that  pictures  can  be  labeled, 
only  a  text  can  explicitly  represent  the  information  that  a 
canary  is  a  songbird.  Conversely,  the  shape  of  a  canary  is 
explicit  in  a  drawing,  film,  or  photo,  whereas  shape  is  generally 
only  implicit  in  text  representations.  Either  a  text  or  a 
graphic  representation  may  be  more  appropriate,  according  to 
whether  one  wants  to  convey  the  information  that  a  canary  is  a 
songbird  or  that  it  has  a  particular  shape. 

Thus,  even  after  the  type  of  knowledge  that  is  to  be 
conveyed  is  known,  it  is  still  necessary  to  choose  an  appropriate 
method  of  conveying  it,  given  the  overall  educational  goal.  In 
this  view,  then,  an  ideal  communication  is  one  in  which  the 
conveyance  needs  dictated  by  a  given  subject  matter  domain  are 
well-matched  to  the  affordances  of  the  representations  used  to 
communicate  them. 

Sketch  of  a  Framework 

Figure  l  is  a  diagram  of  a  theoretical  framework  that 
attempts  to  relate  the  influences  of  the  domain  knowledge  itself, 
the  human  system  architecture,  the  specific  educational 
objectives  at  hand,  known  and  new  representational  conventions, 
and  world  knowledge.  The  framework  takes  as  an  assumption  the 
idea  that,  in  general,  the  information  necessary  to  achieve  a 
criterion  level  of  expertise  in  a  given  knowledge  domain 
eventually  needs  to  be  characterized  in  terms  of  representational 
needs  and  specific  conveyance  methods.  There  are  presently  no 
principled  or  empirically-based  guidelines  from  which  to  do  this. 
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Figure  1.  Theoretical  Framework 


Among  other  things,  expertise  in  a  domain  entails  knowing 
its  vocabulary,  terminology,  facts,  associative  and  causal 
relationships,  laws,  problem  solving  methods,  pattern  recognition 
heuristics  and  algorithms,  and  so  on.  However,  referring  to 
domain  knowledge  in  this  (conventional)  way  does  not  necessarily 
address  specific  conveyance  needs.  Hence,  assuming  that  a 
knowledge  domain  has  been  broken  down  into  coherent  units  that 
are  to  be  presented  in  logical  sequence,  there  still  needs  to  be 
a  domain  parser  that  analyzes  each  unit  (or  lesson)  in  terms  of 
conveyance  needs. 

The  conveyance  needs  then  need  to  be  reformulated  by  an 
information  parser  in  terms  of  the  information  categories  in 
Table  2  (or  some  other  set  of  empirically- justifiable 
categories),  and  recommendations  made  regarding  the  type(s)  of 
representations  best  suited  to  each  need.  Thus,  the 
representation  packages  that  are  the  result  of  this  analysis 
delineate  the  type(s)  of  information  to  be  conveyed,  along  with 
suggestions  about  the  best  methods  to  use  to  do  so.  Several 
representations  may  be  suggested  for  each  topic. 

Ideally,  the  suggested  methods  of  presenting  information 
should  be  implemented  and  analyzed  according  to  the  notion  that 
each  representation  acts  like  an  information  transfer  function. 
That  is,  each  representation  will  emphasize,  pass,  de-emphasize. 
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and  omit  various  aspects  of  the  original  information.  Thus,  the 
representation  analyzer  should  be  able  to  determine,  for  each 
representation,  exactly  what  information  it  will  convey. 

Different  represen  tations  of  the  '’same''  information  can  then  be 
compared  with  each  other  along  dimensions  such  as  ease  of 
acquisition,  interpretability ,  memorability,  original  educational 
goals,  etc. 

It  should  be  obvious  that  the  main  work  of  the  model  is  done 
by  the  two  parsers.  The  domain  parser  has  the  task  of  analyzing 
domains  or  topics  according  to  the  information  needed  to  acquire 
expertise,  while  taking  account  of  the  constraints  imposed  by  the 
human  system  architecture  as  well  as  by  specific  educational 
objectives  that  must  be  considered.  Included  in  educational 
objectives  is  information  about  the  presupposed  level  of 
expertise  of  the  targeted  student  population,  since  this  is  the 
yardstick  against  which  criterion  performance  is  measured. 

The  domain  parser  performs  its  task  with  a  specific  view 
towards  suggesting  aspects  of  the  knowledge  domain  that  might 
lend  themselves  to  particular  conveyance  needs  or  methods.  Under 
normal  circumstances,  domain  knowledge  is  not  characterized 
according  to  this  perspective.  Indeed,  at  best,  domain  knowledge 
is  characterized  according  to  the  conclusions,  or  secondary 
characteristics,  that  need  to  be  acquired.  For  example,  domain 
knowledge  is  often  broken  down  into  that  which  is  declarative 
(e.g.,  facts;  rules;  laws)  and  that  which  is  procedural  (e.g., 
motor  skills;  problem  solving  heuristics  or  algorithms),  yet  this 
particular  breakdown  is  normally  neutral  with  respect  to 
representational  issues.  Thus,  the  domain  parser  has  to  parse 
any  "conclusion  needs"  into  packages  that  are  appropriate  for 
specific  conveyance  methods  and  hence,  for  specific  types  of 
representations . 

It  should  be  noted  that  the  domain  parser  should  be  able  to 
request  more  knowledge  about  the  domain  under  consideration, 
including  knowledge  about  the  relative  importance  of  various 
pieces  of  information.  In  addition,  it  should  be  able  to  request 
modifications  of  educational  objectives,  if  current  objectives 
are  perceived  as  difficult  or  impossible  to  implement. 

The  information  parser  categorizes  each  piece  of  knowledge 
from  the  domain  parser  according  to  its  probable  information 
type.  Sometimes,  two  or  more  types  of  representations  might  be 
suggested  for  a  given  piece  of  knowledge.  For  example,  suppose 
the  information  that  a  water  molecule  is  composed  of  two  hydrogen 
atoms  and  one  oxygen  atom  is  to  be  conveyed.  The  formula  2H*  + 

O"  =  HjO  represents  this  information  as  a  quantitative 
relationship;  there  is  an  equality  between  the  valences  of  the 
three  ions  on  the  left  and  the  molecule  on  the  right.  Underlying 
this  quantitative  relationship,  however,  is  a  structural  one 
involving  the  numbers  of  electrons  that  are  permitted  to  be  in 
orbitals  at  different  mean  distances  from  a  nucleus.  It  is  not 
clear,  a  priori,  whether  the  formula  representation  or  a  graphic 
representation  of  the  atomic  structure  is  more  appropriate  her^. 
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so  both  might  be  suggested. 

The  information  parser  should  also  have  a  category  for  new 
graphical  conventions  that  might  need  to  be  learned.  To  be  able 
to  discern  such  new  conventions,  and  to  facilitate  suggestions 
for  representations,  the  information  parser  must  have  access  to 
information  regarding  representational  conventions  in  general. 

It  also  must  have  world  knowledge  (at  a  level  appropriate  to  the 
targeted  student  population) ,  to  be  able  to  determine  what 
structural  and  functional  analogies  might  be  appropriately  used 
in  the  representations  it  suggests. 

The  information  conveyed  by  each  representation  can 
ultimately  be  compared  to  the  original  domain  knowledge,  to  the 
educational  goals,  to  the  criterion  performance  of  some  targeL 
population,  etc.  As  more  is  learned  about  the  success  with  which 
various  representations  convey  different  types  of  information, 
this  knowledge  can  be  input  back  into  the  system  as  further 
constraints  on  both  parsers. 

It  should  be  obvious  that  the  necessary  empirical 
information  to  create  a  working  implementation  of  either  the 
domain  parser  or  the  information  parser  is  missing  from  the 
literature.  However,  it  is  hoped  that  the  current  framework  can 
provide  a  guideline  for  identifying  the  type  of  data  that  would 
be  useful  to  acquire  in  the  service  of  such  an  implementation. 

An  Example  From  Biology 

Suppose  that  one  lesson  in  a  typical  university  introductory 
biology  class  is  centered  around  the  structure  of  DNA  and  how  it 
replicates,  and  that  the  following  information,  which  has  been 
distilled  from  an  introductory  biology  textbook  (Alberts,  Bray, 
Lewis,  Raff,  Roberts,  &  Watson,  1983),  is  the  input  to  the  domain 
parser; 

The  DNA  molecule  is  a  two-stranded  polymer  composed  of  four 
different  nucleotide  bases:  adenine  (A),  cytocine  (C) ,  guanine 
(G) ,  and  thymine  (T) .  Specific  hydrogen  bonding  between  G  and  C 
and  between  A  and  T  causes  complementary  base  pairing  in  which 
each  member  of  a  base  pair  is  located  on  opposite  sides  of  the 
two-stranded  helix  which  is  the  DNA  structure.  The  nucleotide 
bases  are  located  on  the  inside  of  the  helix. 

DNA  replication  entails  a  separation  of  the  double  helix, 
with  each  strand  acting  as  a  template  for  the  formation  of  a  new 
molecule.  Nucleotides  are  added  to  the  parent  template 
sequentially  by  a  process  that  requires  them  to  form 
complementary  base  pairs.  Daughter  strands  are  complementary  in 
sequence  to  their  parent  template  strand,  so  that  each 
replication  duplicates  the  genetic  information  entirely. 

This  pi  'icular  set  of  facts  is  declarative  in  nature,  and 
the  domain  pc.i.ser  might  choose  to  represent  them  using  a 
propositional  format.  The  job  of  the  domain  parser  is  to  break 
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this  information  up  into  units  that  have  coherence  and  yet  that 
are  amenable  to  being  described  in  terms  of  conveyance  needs,  or, 
better  yet,  information  types.  For  example,  if  the  information 
above  were  passed  through  the  parser,  it  might  be  broken  up  into 
smaller  parcels,  such  as: 

(a)  The  DNA  molecule  is  a  two-stranded  polymer  that  is  shaped 
like  a  double  helix. 

(b)  The  molecule  is  composed  of  four  different  subunits: 
adenine  (A) ,  cytocine  (C) ,  guanine  (G) ,  and  thymine  (T) . 

(c)  There  is  specific  hydrogen  bonding  between  G  and  C  and 
between  A  and  T  (i.e.,  the  members  of  a  pair  "fit 
together”  molecularly) .  This  is  called  "complementary 
base-  pairing."  (d)  The  nucleotide  bases  are  located  on 
the  inside  of  the  helix,  with  each  member  of  a  base  pair 
on  opposite  sides. 

(e)  To  replicate,  the  double  helix  must  separate  into  two 
strands . 

(f)  Each  parent  strand  acts  as  a  template  for  the  formation 
of  a  new  DNA  molecule. 

(g)  Nucleotides  are  added  to  the  parent  templates  one  at  a 
time  by  a  process  that  forms  complementary  base  pairs. 

(h)  Daughter  strands  have  a  nucleotide  sequence  that  is  the 
complement  of  their  parent's,  so  that  each  replication 
duplicates  the  genetic  information  entirely. 

We  shall  further  suppose  that,  much  as  the  original 
information  is  presented  in  two  paragraphs,  the  parsed 
information  will  be  output  in  two  information  packages,  one 
containing  points  [a]  through  [d],  and  the  other,  points  [e] 
through  [h]).  There  will  be  suggestions  within  each  information 
package  regarding  conveyance  needs.  For  example,  in  the  unit 
consisting  of  facts  [a]  -  [d],  there  is  an  obvious  need  to  convey 
information  about  an  event.  The  facts,  together  with  the 
suggested  conveyance  needs,  are  passed  on  to  the  information 
parser. 

Given  such  strong  hints,  the  information  parser  would 
corroborate  the  suggestions  of  the  domain  parser,  and  identify 
several  additional  primary  information  types  within  each  of  the 
two  information  packages.  In  the  current  example,  there  may  be 
information  to  be  conveyed  about  new  representational  conventions 
(e.g.,  the  chemical  notation  for  molecular  bonding),  as  well  as 
appearance  and  Identity  information  (the  shape  of  the  double 
helix;  the  molecular  structure  of  the  bases) ,  spatial  relation 
information  (the  location  of  the  nucleotides  on  the  strands; 
complementary  base  pairing) ,  and  event  information  (separation  of 
the  parent  strand;  sequential  addition  of  complementary 
nucleotides  to  the  daughters) . 
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The  information  parser  might  also  identify  and  include  in  its 
output  an  analogy  or  metaphor  for  one  or  more  of  the  information 
types,  based  on  its  world  knowledge.  For  example,  it  might 
suggest  that  a  spiral  staircase  could  serve  as  an  appearance 
analogy  for  the  shape  information,  a  ''lock  and  key”  metaphor 
could  serve  as  a  functional  analogy  for  the  complementary  base 
pairing  idea,  and  a  zipper  could  serve  as  a  process  analogy  for 
the  beginning  step  of  the  DNA  replication  process. 

Finally,  the  information  parser  will  make  suggestions  about 
how  to  best  represent  the  information  it  has  parsed.  In  this 
case,  for  example,  it  might  recommend  that  both  structural  and 
relational  information  about  complementary  base  pairing  (point 
[c])  be  combined  in  a  single  diagrammatic  representation,  partly 
because  the  base  pairing  idea  is  somewhat  complex  and  partly 
because  it  will  involve  relatively  unfamiliar  notation.  The 
remaining  appearance  and  spatial  relation  information  (points 
[a],  [b] ,  and  [d])  might  then  be  combined  into  a  second 
representation . 

Once  the  information  parser  has  made  its  suggestions,  they 
must  be  implemented  in  actual  representations.  The  parser  might 
suggest  several  alternatives  for  implementing  each  of  the 
representations  it  has  suggested.  Until  there  is  more  knowledge 
about  the  information  affordances  of  different  types  of 
representations,  it  will  probably  be  useful  to  implement  more 
than  one  instance  of  each  suggestion,  and  to  make  a  choice  based 
on  the  output  of  the  representation  analyzer.  The  representation 
analyzer  should  allow  selections  of  which  representatives  to  use 
to  be  based  on  the  specific  information  that  is  conveyed  by  each. 

To  illustrate  how  graphic  representations  of  the  same  subject 
matter  portrayed  in  the  same  physical  media  with  the  same 
relative  level  of  realism  can  nevertheless  afford  quite  different 
information,  I  would  like  to  end  this  section  with  some  examples 
of  figures  that  are  meant  to  convey  some  or  all  of  the 
information  stated  in  points  [a]-(d]  above.  Though  all  of  the 
figures  share  a  lot  of  "surface”  similarities,  they  nevertheless 
do  not  afford  the  same  data. 

Figures  2-5  show  four  different  ways  to  represent  some  of  the 
ideas  inherent  in  the  concept  of  complementary  base  pairing,  and 
Figures  5-7  show  different  representations  of  the  structure  of 
DNA.  All  of  the  figures  are  black  and  white  schematic  line 
drawings  with  at  most  two  additional  colors  used  for 
highlighting.  I  wish  to  make  no  judgments  about  the  relative 
merits  of  these  figures;  indeed,  that  is  an  empirical  issue  to  be 
decided  in  the  context  of  information  about  particular 
educational  goals,  student  aptitudes,  other  simultaneously 
available  representations  (e.g.,  text),  etc. 
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(Adapted  from  FateaataM,  O.  lAtS.  DNA. 
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FIgara  2.  DNA  •  Hydragaa  Bead. 
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(Adapted  from  Purvas.  W.K.,  A  Oriana,  G.H.  1983. 

Ufa:  Tha  Selanea  of  Blaiaaw.  Boaton,  MA:  Willard  Grant  Praaa.) 


Pigura  4.  DNA  •  Molacular  Bonding. 


FIgur*  5  Adaptad  from  Albarta,  B..  Bray,  D.,  Lawla,  J.,  Raff,  M.,  Robarta, 

K.,  &  Wataon,  J.D.  (1983).  Molacular  Blologv  of  tha  Call.  NY:  Garland  Publlahing  Co. 

Figure  5.  Double  Helix  Structure 
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In  terms  of  information  conveyed.  Figure  2  depicts  only  the 
two  base  pairs,  but  uses  chemical  bonding  notation  and  is  thus 
specific  with  respect  to  the  molecular  structure  and  the  location 
and  type  of  hydrogen  bonding  between  bases.  In  contrast.  Figure 
3  omits  these  details  by  representing  the  bonding  schematically, 
while  at  the  same  time,  it  places  the  base  pairs  in  the  context 
of  a  portion  of  the  DNA  molecule  as  a  whole.  Thus,  although  the 
specifics  of  the  hydrogen  bonding  are  omitted  in  Figure  3,  some 
additional  structural  information  is  conveyed.  Figure  4,  on  the 
other  hand,  conveys  a  combination  of  the  information  from  Figures 
2  and  3,  insofar  as  the  specifics  of  the  molecular  bonding  are 
included  along  with  contextual  information  about  the  structure  of 
DNA. 

Figure  5,  like  Figure  3,  conveys  the  bonding  between  base 
pairs  schematically,  yet  it  also  conveys  information  about  the 
shape  of  a  double  helix  (point  [a]),  and  about  the  location  and 
identity  of  the  nucleotide  bases  (points  [b]  and  [d]).  In 
addition,  it  conveys  information  about  the  specificity  of  the 
bonding  via  a  visual  analogy  that  somewhat  poorly  instantiates 
the  lock  and  key  analogy,  since  it  appears  as  though  A  is  as 
likely  to  bond  with  G  as  it  is  with  T. 

Figures  6  and  7,  like  Figure  5,  are  also  representations  of 
the  structure  of  the  DNA  molecule.  Whereas  both  6  and  7  convey 
some  information  to  the  effect  that  the  molecule  is 
three-dimensional,  Figure  6  emphasizes  the  arrangement  of  the 
atoms  that  comprise  the  DNA  molecule,  whereas  Figure  7  again 
emphasizes  that  pairwise  structure  of  the  bases  that  make  up  the 
double  helix. 
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It  should  be  clear,  from  just  these  few  examples,  that 
relatively  subtle  differences  between  representations  (e.g., 
compare  Figures  3  and  4, and  Figures  5  and  7)  can  nevertheless 
result  in  the  conveyance  of  different  information,  both  in  the 
quantitative  and  qualitative  sense.  We  are  quite  a  long  way  from 
implementing  an  information  or  domain  parser.  Most  previous 
research  has  implicitly  concentrated  on  questions  about  the 
representation  analyzer  and  the  transfer  functions;  that  is,  most 
research  has  compared  the  educational  results  of  learning  from 
different  types  of  representations.  What  is  needed  at  this 
juncture  is  research  that  is  focused  toward  discovering  the 
principles  underlying  the  construction  of  representations  that 
achieve  particular  information  affordances.  In  addition, 
research  is  needed  which  can  give  us  guidelines  for  reformulating 
domain  knowledge  in  terms  of  conveyance  needs. 


General  Conclusions 


The  effective  use  of  graphics  in  education  requires  good 
models  of  the  domains  to  be  learned  as  well  as  good  models  of  the 
learners.  In  particular,  we  need  to  know  what  information  should 
be  conveyed  from  a  domain  to  achieve  specific  educational  goals, 
what  constraints  are  imposed  by  the  system  architecture  of  the 
learner,  what  conventional  and  world  knowledge  can  be  assumed  and 
what  must  be  explicitly  taught,  what  role  individual  differences 
might  be  expected  to  play  in  the  process,  and  what  type  of 
information  is  best  conveyec  by  different  graphic 
representations.  At  present,  %e  have  only  partial  knowledge 
about  any  of  these  factors,  and  in  some  cases,  note  at  all. 

The  proposed  framework  is  an  attempt  to  clarify  the 
interaction  that  can  be  expected  to  occur  between  conveyance 
needs  that  may  be  domain-specific,  representational  methods  that 
differ  widely  in  terms  of  what  they  are  capable  of  conveying,  and 
a  receiver  environment  that  will  change  as  a  function  of  prior 
knowledge  and  current  and  desired  level  of  expertise. 

It  should  be  clear  that  much  research  is  still  required  to 
understand  the  role  of  graphics  in  education.  In  addition, 
research  is  needed  to  validate  the  general  and  specific 
assumptions  of  the  framework.  It  should  be  noted  that  the 
framework  itself  stands  as  a  set  of  guidelines  for  this  research. 
It  also  stands  as  an  exhortation  about  attacking  old  problems  in 
new  ways  that  incorporate  more  about  what  are  known  to  be 
psychologically  relevant  factors. 

It  seems  fair  to  say  that  the  commonly  held  belief  regarding 
the  utility  of  graphic  representations  under  almost  any 
circumstance  needs  to  be  questioned.  Indeed,  it  will  be  useful 
to  identify  those  domains  for  which  graphic  representations  are 
not  necessary  for  the  effective  conveyance  of  information. 
Similarly,  it  seems  fair  to  say  that  complexity  is  not  always  a 
necessary  feature  of  an  informative  representation,  and  that  the 
conditions  which  warrant  simpler  displays  need  to  be  identified. 
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III.  RESEARCH  ISSUES  FOR  AIDA  (Poison) 


Introduction 

In  framing  the  research  issues,  I  decided  I  should  consult 
an  SHE  (Subject  Matter  Expert) .  I  visited  with  Brian  Dallman  of 
the  Air  Training  Command's  Technical  Training  Center  at  Lowry  Air 
Force  Base.  Dallman  is  the  Wing  Training  Applications  Officer 
and  has  approximately  20  years  of  experience  with  computer-based 
training  in  the  Air  Force.  He  identified  five  areas  which  he 
believes  are  of  major  concern  in  the  development  of  computer 
based  Air  Force  training  materials  and  courses: 

1.  Use  of  Visual  Materials 

2 .  Evaluation 

3.  Interactive  Courseware  Design 

4.  Task  or  Instructional  Analysis 

5.  Human  Computer  Interaction 

These  areas  are  either  cost  drivers  (i.e.,  they  are  very 
expensive  to  implement) ,  and/or  they  are  very  difficult  to  do 
well  and  consequently  are  frequently  done  badly.  They  are, 
therefore,  potential  target  areas  for  an  AIDA.  The  first  of 
these  areas  is  addressed  at  length  in  Section  II  of  this  report. 

I  shall  summarize  issues  in  each  of  the  above  five  areas  in  the 
remainder  of  this  section. 


Use  of  Visual  Materials 

Dallman  estimated  that  at  least  50%  of  the  development 
effort  for  computer  based  instructional  materials  is  devoted  to 
graphic  materials  and  despite  that,  visual  materials  are  still 
not  well  utilized.  The  sources  of  the  problems  can  be  from 

a)  Inappropriate  use  of  graphics,  i.e.  using  the  wrong  type  of 
graphic , 

b)  overuse  of  graphics,  i.  e.  using  graphics  when  they  are  not 
necessary  to  convey  the  infojrmation,  and 

c)  underuse  of  graphics,  i.  e.  failing  to  use  graphics  because 
they  are  time-consuming  and  difficult,  when,  in  fact  they  are 
the  best  way  to  communicate  the  information. 

Friedman's  Section  II  of  this  report  very  thoroughly  addresses 
the  issues  of  implementing  automated  or  semi -automated  use  of 
visual  materials  from  a  cognitive  perspective. 

The  following  research  questions,  which  are  drawn  directly 
from  Friedman's  remarks,  are  framed  with  respect  to  graphics  or 
visual  materials  specifically.  However  some  of  the  issues 
pertain  to  any  type  of  instructional  material  in  an  AIDA  type  of 
framework. 
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The  issues  or  research  questions  are  organized  into  three 
categories : 

1)  Cognitive  architecture  issues  that  stem  from  questions 
related  to  how  people  process  information; 

2)  Methodological  issues  related  to  research  in  the 
educational  use  of  graphics; 

3)  Issues  that  arise  from  the  specific  approach  suggested  by 
Friedman  which  corresponds  in  many  ways  to  the  current 
framework  being  proposed  for  the  AIDA  system (s) . 


Cognitive  Architecture  Issues 


Working  Memory.  As  discussed  in  Section  II  (see  Friedman's 
remarks  on  working  memory,  pp.  )  and  in  Volume  I  of  this 

series  (see  Sections  II  and  III  by  Poison  and  Tennyson)  working 
memory  has  a  limited  capacity.  Only  a  few  “chunks"  of 
information  can  be  held  in  memory  at  once.  Learning  is  impeded 
if  the  student  cannot  hold  the  material  being  presented  in 
working  memory  so  it  can  be  processed  and  stored  in  long  term 
memory  or  if  the  prerequisite  knowledge  to  process  the  piece  of 
information  being  presented  exceeds  the  capacity  of  working 
memory.  However  what  constitutes  a  "chunk"  may  vary  with  level 
of  learning  and  expertise  in  a  topic,  topic  domain,  type  of 
presentation  (textual  vs.  graphic),  and  other  factors.  In  giving 
advice  to  an  instructional  designer  or  trying  to  design  an 
automated  instructional  design  system,  what  constitutes  a  "chunk" 
for  a  given  student  in  a  given  domain  becomes  a  crucial  question. 
The  following  research  questions  are  concerned  with  this  issue  of 
chunking  with  respect  to  visual  materials. 

1.  How  should  graphics  materials  be  chunked  for  presentation 
in  a  learning  task? 

a.  How  do  chunking  units  change  with  expertise? 

2.  Are  there  chunking  principles  for  graphics  that  generalize 
across  substantive  domains? 

3.  For  the  same  content,  are  the  chunks  for  graphics  and  text 
the  same?  Are  the  principles  for  determining  the  chunks  the 
same? 

4.  What  kind  of  redundancies  should  be  built  into  a  graphic  to 
facilitate  chunking  (e.g.,  color  coding)? 


Long-term  Memory.  Long-term  memory  can  be  viewed  as  a  "tangled- 
hierarchy"  of  linked  knowledge  structures  (Anderson,  1983) ,  which 
may  include  a  number  of  different  representational  types, 
productions,  propositions,  spatial  representations,  temporal 
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strings,  and  schemata  or  frames.  These  knowledge  representations 
encapsulate  a  variety  of  different  types  of  knowledge  including 
declarative,  episodic,  procedural,  strategic,  contextual,  and 
causal  knowledge  as  well  as  our  mental  models,  situational 
models,  and  general  world  knowledge.  The  declarative  knowledge 
includes  various  types  of  world  knowledge  including  social 
conventions,  conventions  about  information  being  processed  (e.g., 
English  is  read  left  to  right) ,  and  various  graphical  conventions 
such  as  occlusion  signals  depth.  For  new  information  to  be 
readily  accessible  and  useable  it  must  be  integrated  into  the 
previously  existing  long  term  memory  knowledge  structures. 
Following  are  some  research  guestions  concerning  graphics  that 
relate  to  the  long  term  memory  structures. 

1.  What  are  the  set  of  graphical  conventions? 

a.  General  (e.g.,  perspective;  size-distance  relations; 

action  lines) 

b.  Domain-specific  (e.g.,  maps;  blueprints;  schematics) 

2.  Which  conventions  may  be  assumed  to  be  known  by  a  given 
subject  population?  Which  should  be  taught?  When? 

3.  What  properties  of  graphics  can  be  exploited  to  facilitate 
making  analogies  between  known  information  and  new  information 
(e.g.,  atoms  =  solar  system)  so  that  the  new  information  can 
be  more  readily  integrated  into  our  existing  knowledge 
structures . 

4.  How  do  people  integrate  information  presented  graphically 
with  information  presented  in  text? 


Processing  Resources.  It  is  generally  accepted  that  resources 
for  processing  information  are  limited  such  that  if  the  resource 
demands  of  information  processing  exceeds  the  resource  capacity, 
performance  will  be  degraded.  Prior  to  1979  it  was  assumed  that 
all  processes  competed  for  resources  from  a  single 
undifferentiated  pool,  and,  therefore  any  two  types  of  concurrent 
processing  could  potentially  lead  to  performance  degradation  if 
the  demand  for  resources  exceeded  the  capacity. 

In  an  influential  paper,  Navon  and  Gopher  (1979)  proposed  that 
there  are  multiple  types  of  qualitatively  different  resources, 
each  with  a  limited  capacity.  Processes  which  use  the  same  type 
of  resource  can  potentially  interfere  with  each  other  because  of 
competition  for  scarce  resources,  but  processes  which  use 
different  types  of  resources  will  not.  Navon  and  Gopher  did  not 
delineate  the  potentially  necessary  and  sufficient  set  of 
resource  types  that  would  characterize  the  human  information 
processing  system,  which  is  a  short-coming  of  their  model. 

However,  Wicken's  (1984)  and  Friedman  and  Poison  (1981)  have 
each  proposed  possible  sets  of  multiple  resource  types.  In  each 
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model,  different  types  of  coding  or  representation  (verbal  vs. 
spatial)  would  potentially  draw  upon  different  resource  pools, 
which  possibly  has  educational  implications,  wicken's  model  was 
primarily  developed  for  explaining  and  predicting  performance  in 
complex  tasks,  while  Friedman  and  Poison's  model  was  developed  in 
the  context  of  the  cerebral  specialization  issue,  but  all  three 
frameworks  have  focused  on  the  performance  implications  of  these 
models  rather  than  the  educational  implications. 

The  educational  implications  of  these  models  stem  from  the 
issue  of  whether  learning,  retention,  and  utilization  of 
information  is  improved  if  information  is  processed  and  stored  in 
more  than  one  code  (representational  type)  and  the  extent  to 
which  different  presentation  methods  (text  vs.  graphics  or 
auditory  vs.  visual)  result  in  different  types  of  encoding.  Two 
specific  research  issues  related  to  the  development  of  an  AIDA 
are: 


1.  What  kind  of  redundancies  can  graphics  provide  for  learning 
from  text? 

2.  Is  it  more  resource-efficient  for  graphics  to  be  redundant 
or  nonredundant  with  text?  How  might  this  interact  with 
expertise? 


Methods  Issues 

There  are  number  of  different  methodological  issues  which 
need  to  be  resolved  before  research  in  the  educational 
implications  of  graphics  can  advance  significantly.  Friedman  has 
identified  five  categories  of  methodological  issues: 

A.  Stimulus  Equivalence: 

How  do  you  ensure  that  information  in  graphics  and  information 
in  text  are  equated?  How  does  one  represent  the  information 
content  of  a  graphic? 

B.  Encoding  Specificity: 

How  do  you  appropriately  test  what  is  learned  (e.g.,  recall 
measures  may  not  tap  what  was  derived  from  graphical  input)? 

C.  Domain  Parsing: 

How  do  you  "parse”  a  domain  into  appropriate  modules  for 
learning? 

D.  Graphic  Conventions: 

1.  What  are  the  graphical  conventions  used  in  any 
stimulus  display? 

2.  How  can  they  be  identified  and  improved? 
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3.  How  might  they  distort  research  findings  if  they  are 
not  equated  in  research  materials? 


E.  Domain  Specificity: 

How  do  principles  of  graphics  in  information  transmission 
generalize  across  courseware  modules  within  a  domain  and 
across  domains?  Do  graphic  principles  differ  across  different 
types  of  knowledge  (declarative  knowledge  and  procedural 
knowledge,  for  example)? 


Issues  Related  to  Friedmanns  Framework 

Friedman  has  proposed  a  framework  for  an  AIDA  system  which 
would  provide  guidance  on  graphics  and  visual  information.  The 
framework  assumes  that  an  empirically  determined  method  of 
categorizing  graphics  can  be  developed  (e.g.  realistic  to  — 
abstract) .  Further,  it  assumes  that  for  each  type  or  category  a 
determination  of  the  type  of  information  that  it  conveys  can  be 
accomplished.  The  AIDA  framework  proposed  by  Friedman  consists 
of  three  modules:  1)  a  domain  parser,  2)  an  information  parser, 
and  3)  a  representation  analyzer  (Figure  1) . 

The  domain  parser  takes  into  account  information  about:  1)  the 
constraints  imposed  by  the  nature  of  the  architecture  of  the 
human  information  processing  system  such  as  short  term  memory 
limitations,  processing  resource  types  and  limitations,  nature  of 
long  term  memory,  etc.;  2)  the  educational  objectives;  and  3)  the 
domain  knowledge.  In  this  framework,  it  is  assumed  that  the 
domain  knowledge  has  already  been  organized  into  coherent  units 
that  are  to  be  represented  in  a  logical  sequence  which 
presupposes  some  type  of  knowledge  acquisition  system.  The 
domain  parser  analyzes  the  domain  knowledge  in  terms  of  the 
information  that  needs  to  be  conveyed  and  parsed  into  units  that 
arc  appropriate  for  specific  conveyance  methods  and  therefore 
specific  representations. 

The  information  parser  can  be  conceived  of  as  a  "mini-expert" 
which  has  knowledge  about  representational  conventions,  the 
different  categories  of  graphics  and  the  amount  and  quality  of 
information  that  each  type  of  graphic  can  convey.  With  this 
information  available,  the  information  parser  analyzes  the 
information  units  of  the  domain  parser  and  determines  the  best 
representational  type  or  may  suggest  two  or  more  alternative 
representations . 

The  representational  analyzer  then  determines  for  each 
representation  exactly  what  information  it  will  convey.  If  more 
than  one  representation  has  been  suggested  for  a  given  bit  of 
information,  the  information  analyzer  can  point  out  the 
differences  so  that  decisions  can  be  made  on  how  the  differences 
will  impact  ease  of  learning,  educational  objectives,  etc.  It 
should  be  noted  that  Friedman  acknowledges  that  we  do  not 


46 


currently  have  the  necessary  knowledge  to  implement  such  a 
system.  The  following  questions  would  need  to  be  answered: 


A.  Graphics  categorization: 

How  should  graphics  be  categorized  (see  Table  1)? 

1.  Continuum  from  realistic  to  abstract?  or 

2.  Other  possible  categories? 

B.  Information  Content  of  a  Graphic: 

What  sort  of  information  does  each  type  of  graphic  convey  (see 
Table  2)? 

1.  Primary  characteristics  (e.g.,  appearances;  states)? 

2.  Secondary  characteristics  (e.g.,  category  &  event 
information) ? 

C.  Domain  Parser: 

How  do  you  implement  a  Domain  Parser?  When  in  this  process 
should  choices  be  made  about  graphical  representations? 

D.  Information  Analyzer: 

Given  certain  information  transmission  needs  in  a  courseware 
module,  what  type  of  graphic  is  best?  How  can  this  be 
implemented  in  an  "expert"  Information  Analyzer? 

E.  Representation  Analyzer: 

What  information  will  a  given  representation  transmit,  omit, 
enhance  or  subtract  from?  How  can  a  Representation  Analyzer 
be  implemented  to  enable  this  "transmission  profile"  to  be 
identified? 


Evaluation 

The  issue  of  Evaluation  has  two  aspects.  The  first  aspect  is 
probably  better  termed  "technology  assessment”  (Baker,  1988)  and 
is  concerned  with  the  evaluation  of  the  system  or  its 
subcomponents.  While  the  AIDA  project  should  include  plans  for 
how  assessment  will  be  achieved,  the  research  issues  of 
technology  assessment  are  beyond  the  scope  of  the  project. 
However,  research  issues  of  computer  based  training  systems 
technology  assessment  are  within  the  scope  of  the  Training 
Systems  Division.  Baker  (1989)  has  made  the  case  for  the 
importance  of  this  issue  quite  nicely. 

The  other  aspect  of  evaluation  comes  under  the  heading  of 
student  assessment  or  diagnosis.  In  even  simpler  terms  it  can  be 
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labeled  "testing”.  The  issue  of  how  to  give  design  guidelines 
for  student  assessment  or  how  to  automate  the  testing  process  has 
been  notably  absent  from  any  discussions  to  date,  but  is  a  key 
part  of  the  instructional  process  and  one  which  novice  or  even 
intermediate  level  instructional  designers  have  a  great  deal  of 
difficulty  doing  well,  particularly  in  a  computer  based  training 
environment . 

Student  Assessment  is  an  issue  which  I  think  could  well  be 
approached  in  the  same  manner  as  was  the  use  of  visual  materials. 
Kurt  VanLehn  (1988)  has  done  an  excellent  job  of  identifying  the 
issues  and  methods  of  student  diagnosis  for  intelligent  tutoring 
systems,  but  it  is  my  understanding  that  for  the  near  term,  AIDA 
is  conceived  as  an  advisor  for  more  traditional  computer-based 
training  systems.  A  commissioned  or  in-house  study  could  be  done 
which  reviews  the  problems,  frames  the  issues  in  terms  of 
cognitive  processes,  and  then  outlines  a  framework  for  automating 
the  process.  From  that  paper,  the  detailed  research  issues  which 
need  to  be  solved  in  order  to  automate  student  knowledge 
assessment  can  be  identified. 

For  the  near  term  needs,  the  study  should  identify  the 
existing  methods,  determine  the  types  of  knowledge  for  which  they 
are  appropriate,  rate  or  evaluate  their  effectiveness  and  assess 
the  extent  to  which  they  can  be  formalized  as  a  set  of  rules  for 
a  mini-expert  system  so  that  the  project  managers  can  assess  what 
is  feasible  to  implement  at  this  time.  My  interpretation  of 
Friedman's  remarks  in  Section  II  is  that  automation  of  the  use  of 
visual  materials  is  not  feasible  at  this  time,  nor  are  there  many 
principled  guidelines  that  can  be  used  in  a  graphics  advisor.  It 
is  an  open  question  as  to  the  extent  to  which  this  is  true  for 
student  assessment. 


Interactive  Courseware  Design 

The  cognitive  science  perspective  on  the  educational 
process  emphasizes  the  active  acquisition  of  knowledge  structures 
by  the  learner  (see  Volumes  1  and  2  in  this  series) .  The 
objective  of  having  courseware  or  systems  which  engage  the 
learner  in  processing  the  information  being  presented  is  to 
facilitate  and  encourage  the  learner  to  actively  process  the 
information  so  that  it  becomes  integrated  into  her  existing  long 
term  memory  knowledge  structures.  While  this  is  a  desirable 
goal,  the  typical  novice  computer  based  courseware  developer,  who 
is  accustomed  primarily  to  a  classroom  lecture  mode  of 
instruction,  probably  has  few  notions  of  how  to  accomplish  this 
goal.  Because  of  their  inherent  interactive  nature,  this  is  not 
an  issue  with  intelligent  tutoring  systems,  although  the  systems 
do  vary  along  this  dimension. 

The  AIDA  aspect  of  this  issue  can  be  approached  in  the  same 
manner  as  recommended  for  student  assessment,  using  the 
Friedman's  remarks  as  a  model.  As  recommended  for  the  student 
assessment  issue,  the  study  would  need  to  assess  what  currently 
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could  be  automated,  as  well  as  identify  the  research  issues  and 
the  nature  of  a  system. 


The  second  approach  to  this  issue,  and  which  I  want  to 
advocate,  is  not  specifically  tied  to  an  AIDA,  but  is  a  more 
general  issue  in  the  use  of  computers  in  instruction  (this  is  my 
look  to  the  future  pitch) ,  My  suggestion  for  a  direction  of 
research  is  to  investigate  alternative  ways  to  use  computers  in 
education  which  have  the  objective  of  actively  engaging  the 
learner  in  the  task.  By  this,  I  do  not  mean  "intelligent" 
systems  in  the  sense  that  they  have  an  expert  model,  a  student 
model,  etc.  For  instance,  the  use  of  cooperative  groups  in 
computer-assisted  instruction  is  one  possible  research  area  (see 
Kintsch's  principles  6  and  7  in  Volume  2  of  this  series) . 

Another  area  is  the  use  of  the  computer  as  a  "scaffolding"  or 
support  tool  to  reduce  short  term  memory  load  and  let  the  student 
explore  various  solutions  to  a  problem  that  she  could  not  have 
accomplished  without  support.  These  types  of  tools  encourage  the 
learner  to  reflect  upon  her  own  problem  solving  processes. 

Burton  (1988)  labels  these  "empowering  tools."  The  algebra  tutor 
being  developed  by  W.  Kintsch  and  his  students  is  an  example  of 
such  a  system  (W.  Kintsch,  1989;  Nathan,  1989) . 


Task  or  Instructional  Analysis 

Of  the  five  areas  of  concern.  Task  Analysis  is  the  only  one 
which  has  received  any  emphasis  in  the  AIDA  discussion  and  papers 
to  date.  This  is  an  extremely  critical  area  since,  if  it  is  not 
done  correctly,  it  will  be  difficult,  if  not  impossible,  to  do  a 
first  rate  job  on  the  rest  of  the  instructional  task.  I  am  not 
going  to  try  to  address  the  research  issues  here  since  they  are 
being  addressed  by  others  (see  for  example  the  chapter  by 
Tennyson  in  Volume  5  and  Merrill ^s  remarks  in  Volume  6) . 

However,  I  will  point  out  that  the  Domain  Parser  in  the  Friedman 
system  which  was  discussed  earlier  would  be  a  part  of  any  module 
to  do  a  task  or  instructional  analysis  and  the  same  research 
questions  apply. 

As  a  word  of  caution  about  how  extremely  difficult  and 
expensive  implementing  this  aspect  of  an  AIDA  can  be,  I  will 
point  out  that  the  rather  ambitious  and  expensive  tri-services 
project  for  automating  knowledge  acquisition  met  with  only 
limited  success.  Automating  knowledge  acquisition  for  an 
instructional  system  is  even  more  difficult  than  automating 
knowledge  acquisition  for  an  expert  system  because  of  the 
necessity  for  "decompiling"  the  knowledge  of  the  expert  and  the 
necessity  for  "propaedeutic"  representations  (Halff ,  1988) . 

While  task  analysis  is  amenable  to  the  same  approach  as  used 
for  visual  materials,  I  hesitate  to  recommend  it  until  the 
results  of  this  cycle  are  completed.  If  sufficient  details  about 
how  this  should  be  done  are  gathered  in  this  cycle,  it  may  not  be 
necessary.  Also  research  on  other  parts  of  the  system  could  be 
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conducted  without  having  the  task  analysis  aspect  of  the  system 
functional.  One  or  more  test  data  bases  could  be  implemented  in 
which  the  task  analysis  had  been  done  in  the  traditional  manner. 


Human  Computer  Interaction 

The  fifth  and  final  area  of  concern  has  at  least  two  aspects. 
The  first  is  the  traditional  interpretation  of  this  area:  How  do 
you  construct  the  interface  to  an  instructional  system  such  that 
the  interface  itself  facilitates  rather  than  impedes  the 
instructional  process?  Miller  (1988)  has  addressed  these  issues 
for  Intelligent  Tutoring  Systems  and  the  majority  of  what  he  has 
to  say  also  applies  to  any  computer  based-training  system.  As 
with  the  technology  assessment  issue,  I  don't  believe  the 
research  aspect  of  this  issue  falls  within  the  scope  of  this 
project.  In  contrast  to  the  technology  assessment  area,  I  don't 
believe  it  even  falls  within  the  domain  of  the  Training  Systems 
Division.  Designing  a  system  which  gives  design  advice  for 
interfaces  is  as  least  as  complicated  as  designing  an  AIDA. 
However  as  a  design  issue  it  needs  to  be  taken  into  account  when 
developing  the  specifications  for  an  AIDA  system.  The  interface 
of  AIDA  should  facilitate  the  instructional  design  process,  not 
impede  it. 

There  is  a  second  aspect  of  this  area,  which  I  do  believe  is 
very  relevant  to  this  project,  both  as  a  research  issue  and  as  a 
design  issue.  How  do  you  decide  what  aspects  of  a  task  the 
computer  should  do  and  what  aspect  the  human  should  do?  In  terms 
of  the  design  of  computer-based  instruction,  it  is  a  question  of 
how  much  control  the  computer  should  have  over  the  learning 
process  and  how  much  control  the  human  should  have  for  optimal 
learning  and  transfer.  In  terms  of  implementing  an  AIDA,  the 
question  becomes  what  should  we  have  the  computer  do  and  what 
should  the  instructional  designer  do?  I  feel  that  there  has  been 
little  consensus  on  that  issue  to  date.  However,  making  that 
decision  is  absolutely  necessary  before  any  specifications  can  be 
developed. 
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IV.  CONCLUSION  (Spector) 


Perhaps  it  is  no  surprise  that  there  are  so  many  unsolved 
problems  with  regard  to  the  automation  of  instruction.  After 
all,  what  we  know  about  the  mind  and  how  people  learn  is  still 
somewhat  limited  even  after  a  half  century  of  intensive  research. 
We  know  that  people  apparently  store  textual  information 
differently  than  they  store  visual  information  (Anderson,  1983) . 
However,  we  cannot  say  definitely  how  the  storage  takes  place, 
nor  can  we  identify  the  specific  physiological  structures 
involved  in  the  process.  In  short,  we  do  not  know  how  to  program 
brains.  That  we  should  expect  to  program  instruction  with  great 
efficacy  is  perhaps  premature. 

What  can  be  done  is  to  develop  principled  mechanisms  to 
explore  the  questions  raised  in  this  report  and  elsewhere  (Glaser 
&  Bassock,  1989) .  An  Advanced  Instruction  Design  Advisor  that 
provides  automated  guidance  to  courseware  designers  and  also 
automates  the  process  of  developing  and  delivering  instruction  to 
students  can  provide  an  important  platform  for  the  kind  of 
research  proposed  in  this  report.  The  advantages  of  such  a 
platform  are  that: 

1.  The  means  of  delivering  instruction  can  be  held  constant 
while  various  instructional  prescriptions  are  implemented  and 
tested ; 

2.  Instructional  prescriptions  can  be  easily  incorporated 
into  an  AIDA  and  tested  because  AIDA  provides  for  the  rapid 
prototyping  of  courseware; 

3.  As  knowledge  about  human  learning  progresses,  new 
principles  and  prescriptions  about  learning  and  instruction 
can  be  incorporated;  and 

4.  The  areas  where  knowledge  is  highly  limited  (e.g.,  how  to 
optimize  the  use  of  graphic  representations,  how  to  optimize 
the  use  of  auditory  representations,  etc.)  can  be  addressed  in 
discrete  modules  and  elaborated  as  knowledge  progresses. 

There  are  surely  additional  advantages  to  be  gained  from 
proceeding  with  the  AIDA  research  program  (e.g.,  the  economic 
advantages  that  may  accrue  if  AIDA  is  successful) .  However,  what 
is  basic  to  the  AIDA  project  is  that  it  is  an  attempt  to  provide 
instructional  design  guidance  to  subject  matter  experts 
responsible  for  developing  effective  computer-based  courseware. 
There  are  some  instructional  strategies  and  tactics  that  are 
common  to  lecture-based  and  computer-based  settings.  However, 
the  computer  provides  an  environment  in  which  new  instructional 
methodologies  can  be  implemented  (e.g.,  interactive  and 
individualized  simulations,  adaptive  testing,  learner  controlled 
exploratoriums,  and  so  on) .  To  take  full  advantage  of  these  new 
possibilities,  the  research  proposed  herein  should  proceed  along 
with  the  continuing  work  to  develop  an  AIDA. 
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